Re: [Orekit Developers] Is serialization of propagators really useful

[Luc Maisonobe <Luc.Maisonobe@c-s.fr>]

Le 16/11/2012 11:23, MAISONOBE Luc a écrit :
> Evan Ward <evan.ward@nrl.navy.mil> a écrit :
> 
>> Hi all,
> 
> Hi Evan,
> 
>>
>> MAISONOBE Luc wrote:
>>> There are two reasons for the current behaviour for
>>> FactoryManagedFrames. The first reason was that we really need to
>>> have a single root hierarchy, so when deserializing a frame we have
>>> to plug it in the existing frame, we cannot let it create an
>>> independent tree by deserializing all its ancestors. The second
>>> reason was that we wanted to avoid creating too many copies of
>>> similar frames. These instances are not singleton per se (and don't
>>> need to) but they have some characteristics of singleton, only for
>>> memory consumption purposes. However, since now the data are hold by
>>> TimeStampedCache which is not serializable and the same cache can be
>>> shared by many instances (even in different thread, as it was created
>>> specifically for this purpose), the second reason about memory
>>> consumption problems could be ignored. So you are right, it is worth
>>> looking at what we want to save.
>> I think the single root hierarchy is a good design. The issue with
>> serialization is that it provides "hidden" constructors that are often
>> overlooked. I've attached a small program that creates a second root
>> frame using serialization. Interestingly, it works even with a
>> SecurityManager in place. I got the idea from Joshua Bloch's chapter
>> on Serialization in Effective Java, which explains this and other
>> problems with serialization. I think the proxy objects strategy that
>> you suggested would fix this problem as long as readObject() throws an
>> exception.  The proxy strategy might make it harder to serialize
>> subclasses because they will repeat their parents serialized form in
>> addition to their new information.
> 
> Your example is really interesting. A side effect of the new
> implementation of findCommon implies that this example does not trigger
> an NPE anymore even when two different tree roots are present. In fact,
> the common frame is considered to be the parent of the two different
> GCRF, which is null. So the transform computation goes through all
> frames up to root, but it handles this gracefully!  The reason for this
> unexpected good behaviour is that despite the root has no parent, it has
> a transform from this non-existing parent set up and this transform is
> identity. Hence when we combine all transforms we end up by combining
> the identity between each root and its null parent. So both combined
> transforms from->common and to->common are valid and can be combined
> together.
> 
> Well, I don't think we can rely on this side effect, it's rather ugly...
> 
> I am on the fence about analyzing the rest of the example. Getting the
> two roots implies setting up explicit code for that (picking up the
> frame from within the readResolve method), and I don't think we could
> protect against such practice. As another example, users could reset the
> parent point to null using reflection. Protecting against errors (i.e.
> users getting two roots without explicit coding) is of course the thing
> we should prevent.
> 
> I'll take a look at making sure at least the predefined frames are
> serialized using only a proxy object storing their identifier only, this
> will be a first step towards increasing robustness.

I have fixed the predefined frames serialization and committed the fix
Sunday night. As one of the predefined frames relied on celestial bodies
(the ICRF, which needs solar system barycenter and transitively also
Earth-Moon barycenter), I also had to fix celestial bodies serialization.

> 
>>
>> Thomas Neidhart wrote:
>>> Regarding the serialization of immutable objects: I think Evan made a
>>> good point, that the context in which an object is created (e.g.
>>> which ephemerides loaded) and on which it also depends somehow must
>>> be known when deserializing it again. Looking at the problem from a
>>> pure technical point of view, serializing only the bits and bytes the
>>> object is composed of may be sufficient, but imho it is also very
>>> important to know exactly what the data is all about, so you can use
>>> it correctly for later calculations or analysis.
>>>
>>> We can serialize a CelestialBody, or an Orbit, but how can we ensure,
>>> that the same environment at its creation time is present at the time
>>> of deserialization? And more importantly, how can we indicate that
>>> there is a mis-alignment to prevent improper use of data results
>>> which are very difficult to track down?
> 
> We can't ensure that. However, I see serialization as a simple mechanism
> that can be used for some purposes, with its advantages and drawbacks
> and the user must know what to do. In many cases,
> serialization/deserialization are used together in a known context (for
> example to transfer data in distributed applications) and the meta-data
> should of course be set up accordingly at both sides, either because it
> is transmitted too or because the settings are similar (for example
> external data files like JPL ephemerides and so on).

The fix I did on the frames was done following the principles explained
above, i.e. not saving the pure bytes of the object (for example the
Chebyshev polynomials are not saved) but rather saving the meta-data
(mainly the name for frames, or supported names for the ephemerides
files and name of the body for celestial bodies). So when the object is
deserialized, it is rebuilt using the proper context (or reused if a
singleton with similar context was already built in the JVM).

The user must still make sure the context can be reproduced. Typically
sending the meta-data of a celestial body through some messaging service
from a peer that uses INPOP ephemerides to another peer that uses JPL
files would not work.

> 
>>>
>>> Maybe something like this should not be the goal of a library like
>>> orekit, but then I would remove serialization support completely to
>>> not give the users a wrong impression on the use of it (And I know
>>> that I now sound very much like Gilles ;-).
> 
> I still think serialization for simple classes that are mainly data
> container (orbits, PVCoordinates, attitude ...) are useful (and I know
> that I now sound very much like Luc writing that ;-).
> 
> In fact, we even advertise in the javadoc of IntegratedEphemeris that it
> is serializable, and this was done on purpose. The reason was that the
> ephemeris could be created by one application and then be transmitted
> for reuse by many other connected applications without a need to
> recompute, thus saving computation power.
> 
>>> Imho, shifting away from serializable algorithms is a good idea. I do
>>> not think there exists a use-case where distributed computing can /
>>> shall be achieved by transmitting both function and data.
> 
> So everyone seem to be OK with that part. I see the consensus as : for
> algorithm-type objects, we don't serialize.

Pascal has already removed the serialization for the propagators and so
on. We will merge our work with the most recent commits and push this on
the server soon.

> 
> You can still convince me about data-holding objects, but for now I
> think we have not reached a consensus.

The things I committed yesterday are an attempt to merge the views
expressed so far (i.e. saving few things, taking care of meta-data,
provide only a basic service for low level objects). Of course, it seems
a little over-engineered for celestial bodies, but I think it was worth
doing.

Of course, if you don't agree with this, I can remove it.

best regards,
Luc

> 
>> Perhaps Orekit could acheive the same effect as Java Serialization by
>> adding support for parsing and emitting standardized file formats. For
>> example, an Orbit could be persisted in a CCSDS Orbit Parameter
>> Message. This approach has the advantages that it is release
>> independent, human readable, developers can refer to a standard to
>> understand the context of the data, and object creation always happens
>> through a constructor. The downside is, of course, the extra effort to
>> write parsers and emitters, but this may be comparable to the effort
>> required to implement Serializeable correctly. Orekit already has some
>> support for this with TLEs(read+write) and SP3(read) files.
> 
> This is an interesting point, and in fact support for CCSDS
> recommandations is already registered as a desired feature in the forge
> (see <https://www.orekit.org/forge/issues/13>). If someone wants it and
> is ready to pay for it, we are ready to do it on request.
> 
> best regards,
> Luc
> 
>>
>> Regards,
>> Evan
>>
>>
> 
> 
> 
> ----------------------------------------------------------------
> This message was sent using IMP, the Internet Messaging Program.
>