Re: [Orekit Developers] Database Support Progress

[Bob Reynders <tzbobr@gmail.com>]

Hello Luc and Evan,

Before I am going to answer the questions I will sketch a scenario that I
had in mind when making the design. I think I misunderstood the requirements
a bit.

An Orekit user has an existing database setup using an existing connector, 
my two main focus points were a JPA and a JDBC setup.
There are two possible scenarios for the existing database;
  1. the database is already populated with e.g. EOP data
  2. the database is empty and has to be populated

To fulfill (1) Orekit would have to be flexible enough to plug into the
existing database setup hence the support for the two most used setups;
JDBC and JPA. A user would use the JDBC or JPA modules and _configure_
the existing table and column names to match the EOP fields.

Fulfilling requirement (2) in my design was done by providing a persister
class. I don't think we're too far apart in this case. I saw it as a fully
autonomic class that just had to be instantiated and called in a main method, 
much like the application Luc described. This class would use the same
configuration of fields as in (1) so that both persisting new entries as well
as reading entries would be supported.

With this scenario in mind I will now answer the questions individually. 

> There is one thing that is unclear to me. Your proposal seems to imply  
> that random user will want to do both read and write at the same time  
> (i.e. to persist data by themselves). The read and write aspect should  
> really be separated. In fact, current Orekit *never* writes, it only  
> loads data. so the "write" part should be outside of Orekit, in a  
> separate application, dedicated to initialize or update the database.  
> What should remain in Orekit core is the ability to read (i.e. some  
> improvement of the data loaders for database use).
> 
> So we would suggest three different parts (say three maven or eclipse  
> projects) :
> 
>   - one would be Orekit itself, with new connectors to load from a database
Agreed, in my proposal this would be the 'core' module at the highest level of
abstraction. For convenient JPA and JDBC connectors there would be separate modules.

>   - one would be an application that does implement a connection to
>     a specific database to load from it (i.e. it would be a prototype of
>     what a user who wants only to read from its database would do), of course
>     this part would depend on the former part
Agreed, this would be some sort of demo for the new functionality.

>   - one would be a writer that can populate a database.
Agreed, I put it into 'core' as a utility class but this could be made a 
lot simpler by providing a standalone application.

> I don't understand the configuration part. Do you want to have  
> something flexible enough that simple configuration would allow to  
> switch from one database scheme to another one? This would probably be  
> over-engineered. We don't "switch" from one configuration to another  
> one, but users may need to set up way to connect to their database,  
> and then they will stick on it for a few years (space industry is  
> really conservative). 
The configuration part is definitely not as complex as it appears to be.
I don't want to switch from one database scheme to another but I thought it
had to be possible to plug into an existing database scheme or at least let
the user choose the scheme that was going to be used. My design does _not_
support a user switching from scheme 1 to scheme 2 without doing manual database
interventions.

> So if "configuration" is "develop specific code  
> that implements a general interface", it's OK with us. 
Yes, it is exactly that. I aim to provide default implementations to get a user
up and running ASAP but allow overriding in the form of user specific implementations
of the new interfaces.

> If  configuration is "write an XML/JSON/TXT file that describes the scheme  
> and will be loaded automatically by a magic class that understand  
> everything possible", then it's probably too complicated (and hackish  
> as you write). 
I'm definitely not capable enough to program such a beast, the JPA developers
already did this in the form of 'orm.xml' and I am not a fan.

> Currently, when users need to have a specific loader,  
> they develop it. We provide a set of current ones that meets general  
> needs and use canonical files as they are provided by laboratories  
> (IERS, JPL, ...). When specific ground systems prefer to use their own  
> format, they simply develop code for that, they don't "configure" a  
> general reader.
It seems that I did not explain the configuration enough in my report.
'Configuration' for me meant exactly what you said, implementing an interface or
subclassing an existing class. 
In the case of JPA however, implementing this interface would be rather hackish
itself. To define your own fields in JPA you have to provide either an 'orm.xml'
mapping file or include annotations on a JPA @Entity enabled class.

> So I would suggest to simplify a little the design.
Assuming the requirements are correct I am not sure of what should be simplified.
But if a user should:
  (1) Only be able to point to his database and let Orekit load
from it, it would be a lot easier than having to design something that enables
the configuration (through implementation) of the corresponding scheme, i.e.,
table and field names.
  (2) Not be able to choose or write his own persistence provider, e.g.,
JPA, JDBC or something custom, we can drop the module layout and provide just 
JDBC in the core.

> As per testing, try to select a simple reasonable goal. It's OK to use  
> one implementation (in-memory if you want) and avoid developping yet  
> another layer to isolate everything from implementation.
This layer is already in place with the current design, though I am a fan of not
spending 90% of the development time on unit tests ;)!

Luc, to make sure I understand the requirements properly could you correct 
my scenario or provide one yourself? 

------ 

> I read through your report, and you might want to look at mockito [1]
> (if you haven't already) to help create mocks for testing. I've used it
> writing unit tests for other projects and it makes it easier to test a
> single class at at time. I've even used it to track down a few bugs in
> Orekit itself. Though Orekit only has JUnit as a test dependency, you
> could add mockit as a test dependency to the new maven projects you create.
Thanks for the reference Evan! I am aware of mockito and have used it several
times before to reduce mocking implementations to a bare minimum, I will definitely
use it in Orekit if mocking proves to be a useful testing tool for the database
functionality.

Thanks everyone for the feedback and my apologies for the wall of text!

Cheers
Bob