Re: [Orekit Developers] OEMWriter Work

["Ward, Evan" <Evan.Ward@nrl.navy.mil>]

So now that Hipparchus is being released I'm sensing that an Orekit
release is coming soon. Before that happens I'd like to implement some
of the items discussed here for OEM writing.

* use Appendable
* streaming interface
* good default mapping Frame, TimeScale to String.
* user customizable Frame, TimeScale to String mapping.
* keep existing interface.
* enable ECEF OEM from an Orekit Propagator

Also I'll change all ephemeris files to use 0.0 instead of NaN for
missing velocities or accelerations because, as Hank pointed out, using
NaN creates problems with the shiftedBy() method. Whether the data is
present can still be determined from the EphemerisFile interface.

I also noticed a couple branches (261 and 262) related to ephemeris
file handling. I'll merge these unless anyone know a reason why I
shouldn't.

Let me know if you want to discuss any of these changes.

Best Regards,
EvanOn Tue, 2017-01-17 at 11:24 -0500, Hank Grabowski wrote:
> For OEMs some other practical changes would be things like the meta
> data fields have to updated as the start/stop times keep marching
> forward/backward through propagation and the handling of the various
> blocks.  
> 
> On Tue, Jan 17, 2017 at 9:35 AM, Evan Ward <evan.ward@nrl.navy.mil>
> wrote:
> > Hi Hank,
> > 
> > On 01/15/2017 01:03 PM, Hank Grabowski wrote:
> > > Sorry, I was out this week so just getting to this.  
> >  
> > No problem.
> > 
> > > 
> > > I agree I'm not a huge fan of the frame and time scale strings
> > > since each file may implement those differently.  Do we maybe
> > > want to convert that to be an internal Orekit type for both of
> > > those and then the respective parser/writers need to write a
> > > converter to their internal implementation?
> >  
> > That was my goal in the getFrame() and getTimeScale() methods that
> > return an Orekit Frame and an Orekit TimeScale. If you have an idea
> > for a better internal Orekit type I'd like to hear it. I don't
> > think it is possible to list every frame or time scale that could
> > be used in an ephemeris so I didn't create an enum.
> > 
> > > 
> > > On the on-the-fly ephemeris generation idea I'm not sure how to
> > > proceed on that. I used to do a lot of logging in the step
> > > handlers but there were artifacts of that process that I didn't
> > > completely like so have generally gone to doing the propagation
> > > and then doing other things with a post-analysis.  I understand
> > > that for very large ephemeris generations the memory problem
> > > could be an issue but I'm not sure how I would want to handle the
> > > streaming.  It's a totally different workflow than the current
> > > design.  With modern systems it takes some very very large
> > > propagations to blow the physical memory or memory stack too so
> > > it's more of a theoretical problem than the first one.  I
> > > definitely would be interested in discussing a way to do that
> > > though.
> >  
> > I have it implemented for some internal ephemeris formats. I think
> > the main change to OEMWriter would be making the BufferedWriter (or
> > Appendable) and the spacecraft meta data instance fields. Then
> > OEMWriter could implement OrekitStepHandler. The init() method
> > would call writeHeader() and writeMetaData() while handleStep()
> > would call writeEphemeris() for each step to write a line of the
> > ephemeris. With a few changes I think it should be able to work. I
> > think the biggest change is usage would be that each OEM file would
> > need it's own OEMWriter instead of the current design where one
> > OEMWriter can write many OEM files.
> > 
> > Best Regards,
> > Evan
> > 
> > > 
> > > Hank
> > > 
> > > On Mon, Jan 9, 2017 at 2:55 PM, Evan Ward <evan.ward@nrl.navy.mil
> > > > wrote:
> > > > Hi Hank,
> > > > 
> > > > On 01/04/2017 10:56 AM, Hank Grabowski wrote:
> > > > > I've applied these suggestions to the 259-
> > > > > OrekitEphemerisPropagator branch, ready for review...
> > > > > 
> > > >  
> > > > Sorry it took so long for me to review. I think the OEM writer
> > > > is a good addition to Orekit. From what I've read it looks like
> > > > there are a few pitfalls that we can address before releasing
> > > > Orekit 9.0.
> > > > 
> > > > When I wrote EphemerisFile I only had parsing in mind so in
> > > > some cases I don't think it will work correctly for writing.
> > > > For example the getFrameString() and getTimeScaleString()
> > > > methods are allowed to return any string or even null. (The
> > > > idea is to return the exact representation used in the file.)
> > > > This creates a problem when attempting to write that value. For
> > > > example reading from a SP3 file and writing to an OEM file
> > > > would produce a file with "REF_FRAME = IGS08", which is not a
> > > > valid frame ID according to Appendix 2 of [1]. Or again,
> > > > OrekitEphemerisFile implements getFrameString() as
> > > > Frame.getName() so when writing a TOD ephemeris in the OEM
> > > > format the file would contain something like "REF_FRAME =
> > > > TOD/2010 simple EOP" instead of just TOD. (This didn't show up
> > > > in the test cases since the GCRF frame is used and GCRF's name
> > > > is always "GCRF".) Since it is not practical to rename the
> > > > Orekit frames to match their OEM names, some kind of mapping is
> > > > needed to associate Orekit frames with their OEM names. Time
> > > > scales and center names likely have similar issues. I think we
> > > > can make the OEM writer conform better to the OEM standard and
> > > > make it easier for our users to use it. My proposal is to
> > > > provide a default mapping for the known common cases and
> > > > provide a mechanism for the user to override the value written
> > > > to the file. The common cases would be those listed in [1] and
> > > > already implemented in Orekit.
> > > > 
> > > > One other issue I've noticed is that the OEM file requires the
> > > > entire file to be buffered in memory. This can determine the
> > > > memory usage for some applications. I think we should integrate
> > > > the OEM writer with OrekitFixedStepHandler to be able to output
> > > > an ephemeris as it is computed without buffering the whole
> > > > thing in memory.
> > > > 
> > > > What do you think?
> > > > 
> > > > Best Regards,
> > > > Evan
> > > > 
> > > > [1] https://public.ccsds.org/Pubs/502x0b2c1.pdf
> > > > 
> >