2020-11-19
Attending: Petr, Seb, Josh, Dom, Simon, Frances, Kevin, June, Erin, Anna, David, Melissa, Will, Chris
Start: 2:00 pm UK
Project Status
(2-3 minutes each)
- IDR
- prod90 released today (idr0088: compound screen, idr0095: assymmetry study)
- NGFF
- Zarr meeting: Matlab, R, Arrow, and
- https://github.com/intake/fsspec-reference-maker (cf. Trevor)
- https://github.com/zarr-developers/zarr-python/issues/556
- making other formats (e.g. OME-TIFF, HDF5…) look like Zarr
- Spec work: http://joshmoore.github.io/ngff (based on bikeshed)
- Demo tomorrow
- Labels: properties & plates
- ZarrReader i.e. showinf/bfconvert from Zarr on S3
- Notebooks
- Releases
- omero-web release: requires some work. Decision by Tuesday
- SA
- Nightshade OMERO server and web upgraded
- Glencoe
- more OMERO.tables work upcoming. could use some feedback from IDR
- Community
- Workshops
- McGill workshop on Tuesday (incl. Will)
- Goeteborg on Thursday
- I2K the week after.
- registration deadline tomorrow
- GBI workshop in January
AOB
(5 min. max; tech. Discussion should be highlighted to relevant people and rescheduled)
Main Topic
(20-25 minutes plus 15 minutes questions max)
- Discussion on data migration with Anna Hamacher (Düsseldorf)
- Seb: how much? Could copy all and delete. Anna: on a per project basis
- Seb: only one target? Or multiple clones? Anna: only one
- Anna: details
- user script for multiple projects and/or screens
- full copy or update
- produce nice url
- background scripts (export via CSV)
- cronjobs for copying “master data” (~10 tables)
- duplicate data on trigger (~80 image-related)
- questions
- move to group per publication per docs?
- need to reallocate event ids?
- completely in python
- use of gallery like IDR
- comments
- Will: gallery is installable. (K/V pairs to filter)
- Petr: user-script is admin only? A: hope to filter
- Josh
- read-only
- Chris: any changes to public copy?
- Seb: think private version is the master copy. perhaps run the same apps (gallery) on the private instance
- Anna: no one should change the public system.
- Chris: you may have a scientist who comes in and wants to change things. Or someone who doesn’t want to move the whole dataset. i.e. question how easy it is to make it generic. Requires people to make organizational decisions up front which is unlikely. (Copying parts of the graph)
- Anna: drive it via tag
- Chris: some people won’t be pedantic about not needing exact copies. and/or may need additional metadata
- Anna: would also publish guidelines for the users on what annotations are needed, quality checks, …
- Petr: expect quality check step to be a burden.
- Petr: in nightshade publications you’d be amazed how different project is differently name (“Figure 1”, “Figure 2”, …). Scientists take care that the data looks nice. Give them service support (chgrp, duplicate, create figures, etc.). Strategy is: do it yourself, but ask when necessary.
- Anna: teaching responsibility, i.e. improve the data. See also talk on figure, and doing things better.
- Josh: linking table somehow?
- Josh: proxy idea
- Anna: people want a copy in case the user deletes something (when moving somewhere else)
- Also protection against hacking
- Josh: static copy
- i.e. multiple versions
- Petr: provenance is asked for everywhere. Can I find out who copied what when (and how many times)? Chris: they say they want that…
- JRS: in IDR we put rules in place that are good for us but also good for the scientists/publications. e.g. updates to a publication are (made) difficult.
- Petr: what is the quality of data that a user can publish? e.g. changes are unlikely. or also “publishing it because want to share with X”
- Josh
- duplicate for IDs
- Anna: less important that the IDs are the same, but the timestamps etc. should all be the same. (Nothing in the public system shouldn’t point to the publication system. The other way around is ok)
- Chris: social/structural barriers (rather than technical) i.e. things that will make no sense.
