This box searches only this space. The box at the upper right searches the entire iPlant wiki.

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Task marked complete

1. Project Report (Wiki/Read the docs | Due Dec. 17th @ Noon)

 

  •  

    Abstract:

    •  

      Overview of project 

    •  

      analysis workflow 

    •  

      computational system 

    •  

      results management

  •  

    Project Description:

    •  

      What is the gantry phenotyping platform

    •  

      What sensors does it have

    •  

      How much data does it produce

    •  

      Why do we need to process data coming from this machine ASAP?

    •  

      Overview of workflow and the main features of it

      •  

        Scalable, flexible, modular, extensible, high performant, high reliability (checks and retries errors)

  •  

    Team Members:

    •  

      Overview of the team and team responsibilities

    •  

      List teams

    •  

      Team lead(s)

    •  

      Team members

    •  

      Role of each person (a couple of words)

  •  

    Input Data:

    •  

      Description of data: which sensors, what do they do

    •  

      How much data

    •  

      Which where 

    •  

      Where/how were they obtained.

    •  

      What was done to clean them for processing.

    •  

      Link, description, and instructions of using code to get and clean data.

  •  

    Analysis Workflow/Code - Workflow team (John:Cloud & Michele: HPC )

    •  

      Description of analytical workflow.

    •  

      Description of computational framework (master/worker with Workqueue and Makeflow)

    •  

      Instructions for use

    •  

      Description of datasets

    •  

      Describe scaling methods

    •  

      Description of how workflow can be modified

  •  

    Results - Jiatian & Jialiang Wang

    •  

      How are results managed

    •  

      How are results evaluated for faults

    •  

      Where can researchers find the results

  •  

    Project plan and timeline - Sateesh & Emmanuel

  •  

    Detailed benchmarks - Jiatian wang & Jialiang Wang

    •  

      How long it took to run the full dataset:

    •  

      Software installation

    •  

      Data Staging

    •  

      Data Processing

    •  

      Workflow monitoring

    •  

      Results deposition

    •  

      How much would this cost to do on AWS (e.g., for each workflow, approximate cost to process one day’s worth of data)

  •  

    Post-Mortem Analysis (SubPage) Documentation

    •  

      One for each team!

    •  

      Focus on team processes; not technical problems

    •  

      What worked well

    •  

      What didn't work well

    •  

      What you would do differently

2. Project Code  (GitHub) and Documentation (Read the Docs)

  •  

    Overview

  •  

    System requirements

  •  

    Getting Started

  •  

    How to use

  •  

    How to scale (what is needed)

  •  

    Description of output

  •  

    Warnings and caveats

  •  

    License

  •  

    Author description

  •  

    Where do get additional help

3. Client Presentation (In-Class | Due Dec. 17 @ 8AM)

  • Introduction (Sateesh Peri)

  • Overview of project

  • Overview of workflow

  • Overview of solution

  • Overview of results

  • Details on:

    • Data: obtaining, cleaning, about the final dataset.

    • Analysis workflow: overview, obtaining, how to use

    • Benchmarks: Scaling, full analysis time

    • Results and result management

  • Code:

    • Where is it?

    • What does it take to rerun it?

    • Show an example of making a modification to workflow (adding a new extractor) an rerunning

  • Documentation and training materials

    • How would a new person take the code and get to work?

  • Live demo (John Xu & Class)

    • Show us the system in action!

  • Conclusion

    • You have done some amazing work, here is your chance to put a bow on it

  • Thank you

    • Team members, roles and responsibilities 

    • Help from external teams (cctools, David LB group, class clients, CyVerse people, etc.)  Note -- always make sure to give credit (sometimes extra credit) for anyone who has helped make the project possible

    • Computational infrastructure:  UA HPC, CyVerse, XSEDE, JetStream

4. Teammate Evaluations

  • You will lose 50% of your final score if this is not completed. 

  • This is for each sub team

  • https://forms.gle/3QY1cWVowm1PYSsE6

  • Note: Feel free to review team members in other teams (especially if someone has gone above and beyond to help)