OME at LOCI –
Roadmap and future directions
Here is a detailed list of OME-related targets for our software projects
over the next year and beyond. For more information about the future of OME
in general, LOCI's role, and other topics, please see the
Frequently asked questions. For a more thorough,
technical, and up to date list of LOCI software issues and goals, see our
list of Trac tickets.
Better OME web interface launcher (2008).
To launch VisBio from the OME web interface via Java Web Start, a JNLP
document must be generated on the fly. This document contains a list of JAR
files (libraries) necessary for VisBio to launch. Unfortunately, as VisBio
evolves and grows, this list changes. When that happens, old versions of
OME may lose the ability to launch VisBio properly (they generate JNLP
documents with an old list of JARs).
To account for this issue, we will do two things. First, starting with OME
2.6.1, we will bundle a stable version of VisBio that will always work with
that version of OME. Second, we will introduce a JNLP template on the LOCI
web site that the OME installation can access, which indicates the proper
form the JNLP document should take in order to successfully launch the
latest version of VisBio. Lastly, we will provide an option in the
installer to switch between using the bundled VisBio version (served via
HTTP from the local OME server), or the latest version available (served
from LOCI's web site).
[#75]
Further integration with Bio-Formats (2008).
VisBio was written before Bio-Formats existed; in fact, Bio-Formats was
originally split off into its own library from VisBio's growing list of
supported file formats. Bio-Formats development continued on its own, and
eventually VisBio was retooled to use Bio-Formats in its current
incarnation.
But VisBio's file stitching and axis guessing features are still based on
old code, rather than the new and improved Bio-Formats logic. VisBio needs
to be updated to use more Bio-Formats features directly. Doing so will
improve VisBio's axis guessing logic, ensure VisBio stays in sync with
future enhancements made to Bio-Formats, and most importantly will finally
enable VisBio to fully support datasets with multiple internal dimensional
axes.
[#35]
Performance improvements (2008).
VisBio uses VisAD for its displays, which provides incredible flexibility
at the expense of performance in some ways. The latest version of VisBio
uses a new kind of VisAD data renderer tailored toward efficiency with
image planes, but more work can be done to improve VisBio's performance
with datasets both large and small.
[#34]
- For some operations, VisBio transforms data into floating point
representation. Doing so can be avoided in many cases with some additional
coding. In some cases, such as image resampling, transforming between
Java's internal image representation and VisAD data object representation
can be avoided completely.
[#37]
- Right now, VisBio keeps exactly one full resolution image plane in
memory at a time. VisBio needs a more flexible interface for specifying
which planes to cache at once. These improvements would allow VisBio to
more easily provide full resolution animation on demand.
[#49]
- VisBio's thumbnail cache files are inefficient (up to 8 times larger
than they need to be, since they write the thumbnails in floating point
representation). The thumbnail disk caching needs to be reworked. Thumbnail
support may also work better by utilizing Bio-Formats's thumbnail API.
[#46]
- Downloading pixels from an OME database does not currently use the
improved image data renderer, but it should.
[#53]
Explicit support for spectral-lifetime (SLIM) data (2008).
Slim Plotter is becoming increasingly sophisticated. One of the major
original goals of VisBio was to work with data of any dimensionality,
particularly combined spectral lifetime data, but currently Slim Plotter is
a far better tool for doing so because it is specifically tailored to it.
Eventually, Slim Plotter's features will be integrated into VisBio, so that
extra features VisBio provides can be used with spectral lifetime datasets.
But we will not discontinue Slim Plotter until the SLIM-enabled VisBio is
as functional and easy to use as Slim Plotter.
[#61]
Intelligent OME metadata support (2008-9).
E.g., store and retrieve overlays to and from an OME database, or as
OME-XML on disk.
Better integration with MATLAB and other external programs (2008-9).
VisBio provides some basic integration with other software through its
MATLAB and external program data transforms, but many more possibilities
for interoperability exist. Our eventual goal is for VisBio to contain
several full-featured scripting possibilities, similar to ImageJ's macro
and plugin facilities.
Acquire data directly into an OME database
(Q1-2 2007). Mostly done, but disabled for now.
While there are multiple ways to feed newly acquired data to an OME
database, we have chosen to go a slightly indirect, but potentially more
robust, route: saving the data as OME-TIFF, then uploading the files to the
database server via SSH or FTP, triggering an import on the server side
(a small server-side module utilizing
inotify or similar
functionality watches for changes to the file system).
WiscScan currently possesses the ability to perform such uploads, but it is
disabled within our lab until we fully deploy our production OME server.
Once the server is up and running, we will enable automatic import into the
DB.
Our current version of the server-side import function works with the OME
v2.6.x database, rather than the OMERO server. Ultimately, we would like to
deploy OMERO and have the server-side module communicate with that instead.
Lastly, the system (on the server side) is not currently configured to
email the user once the data has been successfully imported; setting up
such functionality will require a small time investment.
Support new hardware such as firewire cameras (2007).
WiscScan is now capable of controlling all LOCI microscopes.
Record more metadata, including custom fields (2008).
We will thoroughly enumerate every aspect of our collection hardware, and
other experimental and user parameters, as OME-XML.
Intelligent OME metadata support (2008-9).
We would like to store and retrieve to and from an OME database, or as
OME-XML on disk, several important pieces of metadata:
calibration information (physical image dimensions), regions/overlays,
color table settings, and textual annotations and notes.
JUnit testing framework, to help prevent regression bugs
(Q4 2006). Done.
We have created a test suite for verifying Bio-Formats import results for
our entire collection of sample data. The suite is extensible to any
collection of test data, and the scope of tests performed can be expanded
by creating helper configuration files describing the expected results for
each data file.
omeul – Command line upload utility using OME's Java Remote
Framework (Q1 2007). Done.
The omeul utility is now available from the Bio-Formats web page as part of
the Bio-Formats command line tools.
[#193]
omebf – Integration with OME server
(2.6.1, 2007). Done.
OME v2.6.1 includes Bio-Formats, and provides an integrated import
mechanism, such that any format supported by Bio-Formats can be imported
into an OME database on the server side.
[#68]
Improve OME-XML conversion for all formats (2008+).
Complete conversion of a single file format into OME-XML is a daunting
task, and Bio-Formats supports over 50 such formats. Ultimately, we want
Bio-Formats to provide a reference implementation for representation of
third-party formats' metadata in OME-XML (i.e., for conversion to OME-TIFF).
Ideally, it would be possible to convert a given OME-TIFF file
back to the original format—we have no plans to write code
to perform such a task, but want to ensure such a thing would be possible
in order to guarantee that no metadata was lost in the conversion
to OME-TIFF. However, many third-party formats contain information
of a legacy nature, unused fields, and other "cruft" that is not necessary
or even desirable to preserve as dedicated OME-XML fields; the mapping of
every such field would be unduly burdensome with no real benefit.
The general solution to this mapping dilemma is to map every piece of
useful metadata to OME-XML (the hard part), then store the remaining fields
in an appendix of "original metadata," unmapped, merely as key/value string
pairs using the original format's field naming scheme where possible. This
technique would preserve all metadata no matter how minor, ensuring that the
OME-TIFF file could be converted back to the original format if desired, but
without undue difficulty of implementation, or bloat to the OME-XML schema.
[#62]
Support for additional third-party file formats (2008+).
As time permits.
New file formats are constantly being introduced. To continue to reap
the benefits of the OME-XML standard, these formats must be fully
convertible with OME-TIFF, so that OME-aware software can interact with them
as effectively as possible. We do not want to encourage third-party
developers to develop additional proprietary formats, but when they do we
have little choice but to add support for them.
[#42,
#44
,
#45
,
#55,
#179,
#180,
#221,
#228]
Split plugins: "Upload to OME" and "Download from OME"
(Q4 2006). Done.
The OME plugin has been revamped to work more reliably with OME 2.6.0
database systems, with several bugfixes and improvements.
Better metadata download, complete metadata upload
(Q4 2006). Mostly done.
The Upload to OME plugin now uploads OME-XML metadata to the OME database.
The Download from OME plugin can download a limited subset of metadata
associated the selected Images. It is still not possible to download the
complete set of metadata associated with a given Image in one fell swoop.
Adding such functionality to the remote framework would probably require
the creation of one or more new XML-RPC method calls on the perl side.
Explicit support for derived pixels—commit changes back to OME
database (2008-9).
The OME system provides support for multiple sets of pixels, including
pixels derived from other pixels. This functionality is ideal for use with
ImageJ, which provides many image processing tools. The eventual goal is to
provide the ability to download pixels from an OME database, perform
image processing operations on the pixels with ImageJ, then save the
changes back to the OME database as a derived set of pixels.
Access server-side analysis and other server-side functionality from the
client (2008-9).
The OME server possesses a rich analysis engine that can in theory be called
from a client machine using the remote framework. We plan to investigate
benefits of invoking the OME server's analysis mechanism from the client,
to enable access to server-side analysis tools from within ImageJ.
API to delete unwanted elements (2008).
Right now, the Java OME-XML library (part of OME Java, in the
org.openmicroscopy.xml package tree) provides the ability to build up an
OME-XML structure from a string, from a file, or from scratch. But there
are currently no convenience methods to delete portions of the tree. Doing
so is not trivial because if a portion of the structure is deleted, there
is a question of whether other parts of the structure that reference the
deleted portion should also be removed.
Providing methods for deletion is not a particularly pressing issue, but
would be nice to include for completeness.
Display data downloaded from an OME server
(Q2-3 2007). Done.
The Download from OME plugin now uses much of the same core Bio-Formats
logic as the normal Bio-Formats Importer plugin, which means that any images
downloaded from OME are prompted with the same options dialog, including the
ability to view the data using the Data Browser (or Image5D, or View5D).
Link virtualization support to OME download ability
(Q2-3 2007). Done.
The Download from OME plugin now uses much of the same core Bio-Formats
logic as the normal Bio-Formats Importer plugin, which means that any images
downloaded from OME are prompted with the same options dialog, including the
ability to view the data using the Data Browser with virtualization,
meaning planes are accessed and cached across the network on demand.
Java Web Start link in OME web interface (2008).
The OME web interface currently has a "View in VisBio" link that spawns an
instance of VisBio via Java Web Start (JWS). It would be nice to have
another such link for ImageJ using the Download from OME and 4D Data
Browser plugins, labeled "View in ImageJ" or some such, that takes advantage
of these plugins' functionality.
Execute ImageJ macros on virtualized data (2008-9).
Currently, performing any command that alters image planes
(e.g., Process>Smooth) affects only the currently displayed image plane.
As long as that plane remains in the cache, it will remain altered, but as
soon as the plane is dropped from memory (e.g., if the user navigates to a
new plane too far away from the original plane), the action is lost, since
the plane is recached from disk.
The easiest solution to this issue is to take advantage of ImageJ's
comprehensive macro language functionality, recording each command as it
happens into a "preprocessing macro" that is dynamically applied to planes
as they occur, and as new planes are cached. To the user, the behavior
would seem nearly identical to a normal image window, except for potential
delays from CPU-intensive image processing operations.
[#57]
Commit changes back to OME server (2008-9).
See Explicit support for derived pixels in the
OME plugins for ImageJ section above.
Editing and saving capabilities (Q1 2007).
Done, but disabled for now.
The ability to edit and save metadata with OME Notes will be enabled
when we are confident the feature is robust, and reasonable safeguards are
in place to prevent saving corrupt OME-XML.
Integration with VisBio (2008).
VisBio provides the ability to view both the original metadata table, as
well as the converted OME-XML, in a tree structure. But that code is out of
date, and OME Notes is much more sophisticated. VisBio should be
retooled so that viewing a dataset's metadata brings up OME Notes.
[#71]
Improved templating system (2008).
We are working on a major overhaul of OME Notes with a more powerful
templating system, so that metadata can be arranged onscreen in any desired
configuration, similar to an actual, physical notebook.
[#72]
We also plan to create a template editor so that users can easily lay out
their notebooks however they want ("design my notebook").
[#73]
We will create several preset templates for various use cases, including
templates for VisBio, WiscScan, the Data Browser, third party
acquisition systems, and standalone.
[#74]
User testing, implementation of additional user requirements (2008).
OME Notes would benefit greatly from additional development cycles to
provide additional features and ease-of-use, to maximize the program's
utility.
Save and load image metadata from OME database (2009).
OME Notes currently operates on OME-XML metadata on disk in either
OME-XML or OME-TIFF formats. It would benefit from the ability to access
metadata directly from an OME database, using the Java remote framework.
Such functionality, however, does depend on our progress with metadata
downloading as detailed under Better metadata download in the
OME plugins for ImageJ section above.
Save OME-XML metadata to OME-TIFF header (2007).
Mostly done.
To convert to OME-TIFF, the Data Converter uses Bio-Formats's built-in
ability to generate an OME-XML metadata block for any input data, then
writes that metadata block to each output TIFF file's ImageDescription,
inserting TiffData elements appropriate to the image plane organization.
The OME-XML block cannot be used exactly as-is, however, as a TiffData
element needs to be inserted. The easiest way to solve this problem is to
simply use the forthcoming OmeTiffWriter, which will take care of most of
the intricacies of OME-TIFF writing under the hood.
Convert multi-file images from one file orientation or
compression scheme to another (2007). Mostly done.
We have extended the Data Converter from its old incarnation, a
small QuickTime to TIFF conversion utility, to a full-powered Bio-Formats
conversion tool, capable of importing data in any supported format with any
numbering scheme, and saving to any supported output format, most notably
OME-TIFF.
Adapt into an export GUI component within Bio-Formats (Q2 2008).
The Data Converter application is a good start on a general-purpose export
interface. VisBio also has an export function that was created before
Bio-Formats existed. We plan to unify these interfaces in a common set of
classes, and rework both Data Converter and VisBio to use the common
infrastructure.
[#69]
Bin channels together and output SDT files for use in SPCImage
(2008).
SPCImage provides a number of useful analysis capabilities, but mainly
operates on data in SDT format. It is also not particularly well-suited to
processing multi-spectral data. As such, it would be nice if Slim Plotter
could export multiple channels, binned together, to a new SDT file for use
within SPCImage, particularly the curve fitting functionality.
[#28]
Better curve fitting (2008).
Slim Plotter can fit single-exponential curves using the
Levenberg-Marquardt nonlinear least squares algorithm, but would benefit
from improved fit precision
[#60],
as well as multi-exponential fit capability
[#24].
Extend OME-XML to support spectral lifetime images (2008).
OME-XML is currently tailored toward "5-D" data—width and height,
focal planes, time points, and channels—with "channels" providing a
catch-all for any additional dimensions, including both spectra and
lifetime bins. As such, we must minorly extend OME-XML to include a new
imaging modality type for SLIM, and provide a new element to identify the
channel rasterization.
Investigate techniques for SLIM data compression and storage (2008).
SLIM data is interesting because it is often quite sparse (lots of zeroes),
and is also very similar across many dimensions, especially spectral
channel, focal plane, and time. As such, it is highly compressible, though
identifying a compression scheme and file format to represent this kind of
data that is efficient both space-wise (disk usage) and time-wise (quick
extraction of individual image planes) is challenging.
One promising technique LOCI has explored in the past is lossy
multidimensional compression using wavelets. We are exploring adapting
these techniques, as well as lossless variants thereof, to SLIM data.
Integrate features into VisBio (2009).
See Explicit support for spectral-lifetime (SLIM) data in the
VisBio section above.
[#61]
Last update: Wednesday, January 23, 2008
