2014.01.16 BIEN db

Owned by narro
Last updated: Feb 05, 2014Version comment

BIEN Database

January 16, 2014

Participants

Aaron, Brad, Brian, Ramona, Martha

Agenda

Review progress on tasks
Discuss the need to led Aaron and Brad focus on getting db work done

Previous Week's To Do
Aaron

  • Item 2. "Plot data providers" from Brad's "High priority tasks" email message of Dec. 17/18.
  • GBIF
    • Taxon names (in Jan. for beta)
    • Code a workaround for the accepted names that are missing family names.
    • Then, re-run the names that are missing family names.
  • BIEN2 Traits (in Jan. for beta)
    • Aaron will left join "the view that left joins all the tables" to the additional trait information for Brad to validate.

Martha

  • Remind Brian about the SALVIAS publishable data issue. (Reminded him.)
  • From last week: Add to the BIEN development plan under serving data > API > interface,
    • Set things up so that a user could search TRY simultaneously when searching BIEN.
    • Then, user could request any TRY data that is not already in BIEN.

Notes

Plot and Project Data Providers 

  • Aaron mapped project contributors from VegBank to both VegBIEN and VegCore.
    • The data are in VegBIEN.
    • So it's done for VegBank.
  • Next, Aaron needs to do the same for Salvias.
    • Estimated completion: by Monday or Tuesday.
    • Refresh takes a day. Requires active attention.

GBIF taxon names (completed)

BIEN2 traits

  • Traits are in the normalized “trait” table and in the ______ view.
  • Aaron - ERD needs to be updated. Later. 
    • Good practice is to update ERD as we go.
  • Aaron needs to add columns to the denormalized view. They come from a left join.
  • Decision: Take this off Aaron’s plate.
  • Brad will do it using the normalized trait table and the oringinal input data from BIEN2.
    • Brad: Spot check the data.
    • Brad: Write quantitative validations.
    • Brad will do it by M or Tu and send it to Aaron.
    • Aaron: Put Brad’s quantitative validations into the validation pipeline.

Spot checking issues

  • Clarification: For now, Aaron should not work on the other issues found from spot checking other data sources. That is a lower priority than the Quantitative Validation work.

Quantitative Validation

  • The plan is to create two files and diff them.
  • This work is described in Item 3. "Plot data providers" from Brad's "High priority tasks" email message of Dec. 17/18.
  • The SALVIAS quantitative validation queries represent most queries that will need to be done, so Aaron can subset those as needed for other sources. For item 3.4.
  • For items 3.1 and 3.2: (SALVIAS)
    • As noted in item 3.1 Brad had problems with queries 12, 13, 15 (subplot codes) because couldn’t find the taxon information. Also see Brad’s Dec. 10 email subject: Quantative validations for SALVIAS.
    • Brad has always had concerns about his recursive data are stored. Prefers to use an index to retrieve any level of a recursive store of data.
    • Discussion of how to fix queries 15, 12, 13.
    • Brad will send the queries on the original SALVIAS database, which he forgot to attach to the Dec. 10 email.
    • Aaron will work on items 3.1 and 3.1 under Quantitative validations.
    • After Aaron fixes the queries, send them back to Brad so he understands what needed to be done.

Decisions and Clarifications

Decision: Take validation of BIEN2 traits off Aaron’s plate. Brad will do it.

Clarification: For now, Aaron should not work on the other issues found from spot checking other data sources. That is a lower priority than the Quantitative Validation work.

To Do

Plot (and Project) Data Providers

Aaron: For SALVIAS, complete the work as described in Item 2. "Plot data providers" from Brad's "High priority tasks" email message of Dec. 17/18.

BIEN2 Traits

Brad: Validate the BIEN2 trait data, taking this task off Aaron's plate. Use the VegBIEN normalized trait table and the oringinal input data from BIEN2.

  • Spot check the data.
  • Write quantitative validation queries.
  • Send queries to Aaron so he can put them into the validation pipeline.

Aaron: After Brad sends you his BIEN2 Trait quantitative validation queries, put them into the validation pipeline.

Quantitative Validations

Brad: Send Aaron the queries on the original SALVIAS database, which he forgot to attach previously.

Aaron: Work on Items 3.1 and 3.2 described in "Plot data providers" in Brad's "High priority tasks" email message of Dec. 17/18.

  • After the queries (12,13,15) are fixed, send them back to Brad so he understands where his mistakes were.