Re: [Orekit Developers] Work In Progress for thread safety

[Thomas Neidhart <thomas.neidhart@gmail.com>]

On 04/13/2012 05:43 PM, Luc Maisonobe wrote:

> OK, Issue 89 is now fixed. As part of the fix, I have changed dates
> quantization to use long and set up fixed quantum step to 1 microsecond.
> 
> This should avoid overflow, but does not prevent the large number of
> slots and even a large number or calls to the generator 1589 in a test I
> just run!
> 
> So there is definitely something to do to avoid this.

Yes, the other case when slots are (unnecessarily) created is related to
the limiting of the range the generator can provide. Adding this fix as
discussed before should fix it (tested it locally).

> While debugging issue 89, it appears preventing extra calls is not
> trivial. We must make sure we don't prevent generating new entries
> before the first existing one or after the last existing one when there
> is still some room for potential new entries before generator end. So we
> have to compare the current first (or last) entry with generator
> boundaries and then decide what to do.
> 
> We must also take care that in the general case, slots will be evicted,
> so even if some data at boundary has already been generated, it may have
> been evicted and may have to be generated again.
> 
> This may happen for UTC-TAI, as I think it would be more efficient to
> limit the size of the slots to say 1 year, so the search at deepest
> level (in the slot) remains fast.

I have looked into this, and the reason (at least for the UTCScale) that
additional call to the generator are performed is a corner case due to
the neighborSize:

 - consider you have 40 entries and a neighborSize of 2
 - requesting neighbors for a central date that correspond to the last
   entry would require the cache to be further filled, as the request
   can not be fulfilled: lastIndex + 2 > cache.size()
 - thus the generator is asked for new entries
 - the UTCScale Generator can not provide more entries, so there is a
   safe-guard in TimeStampedCache.Slot.getNeighbors that reduces
   lastIndex to fit the available cache size

The problem is that this check is done at the end of the method, after
requesting the generator. To tackle this, two solution come to my mind:

 - do a check at the beginning like this:

   // if we request neighbors outside the available limit
   // and the last entry is already at the generator bound, adjust
   // the index of the first neighbor to return to match the current
   // available cache entries without requesting more data, as the
   // generator will not be able to provide more data
   if (firstNeighbor + neighborsSize > cache.size() &&
       getLatest().getDate().compareTo(generator.getLatest()) >= 0) {
       // we end up with a non-balanced neighborhood,
       // adjust the start point to fit within the cache
       firstNeighbor = cache.size() - neighborsSize;
   }

   similar for the lower bound

 - add a method to the generator interface (maybe a bad name, but to
   get the idea):

   public boolean isSingleShot();

   which indicates that the generator is providing all its data in one
   shot. The Slot.getNeighbors method does a similar check as before:

   // same as above, just do not rely on bounds but use an explicit
   // indication from the generator
   if (firstNeighbor + neighborsSize > cache.size() &&
       generator.isSingleShot()) {
       // we end up with a non-balanced neighborhood,
       // adjust the start point to fit within the cache
       firstNeighbor = cache.size() - neighborsSize;
   }

Using this the multi-threading test runs through with only 1 generate call.

Thomas