[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Orekit Developers] Multi-threading problems in OREKIT



Thomas Neidhart <thomas.neidhart@gmail.com> a écrit :

On 02/22/2012 09:18 AM, MAISONOBE Luc wrote:
Hi Thomas,

Hi Luc,

Yes. I hesitated to do that, as it could increase complexity. If this
occurs, it will occur once and then the capacity of the slot would be
increased to maxSlot + 1, and therefore will never be exceeded again.

What would you think about allocating the slots list with a capacity set
to maxSlots + 1 right from the constructor (it is currently allocated
with maxSlots). The alternative to avoid the additional entry implies
handling the entries shift by ourselves between the new entry and the
evicted one, taking care of which one is in the front since the shift is
either one index forward or backward.

yes, that's what I meant with increasing the initial capacity, but as
you pointed out yourself this is really a very very minor thing.

After looking further, in fact the code is simpler if I make sure there is enough room. It's implemented and committed now.


I understand you point. I considered the case where points are read from
a file (say a JPL ephemeris). In this case, it seemed better to create
bunch of points instead of one point only. However, this could be
handled at generator level and not at cache level. So the cache could
ask for just the next point, and the few generators that know it is more
efficient to read a whole set at once will do it. In fact, the API
already allows it since generators can provide more points than
requested, I simply did not fully used it.

ok this sounds good to me. I guess this will become more clearer once we
have experience with using such a cache for different objects /
scenarios. Do you plan btw. to make the caches configurable? The default
should be hidden / preconfigured for normal users, but it would be good
if one can change the defaults.

I have changed slightly the API and implementation so it is easier to support both cases (generating one point at a time or generating several at once). The generator can now decide by itself what it generates, and the cache will make sure it has enough point, possibly by calling the generator several times.

If you look at the tests now (I have included a slightly modified version of yours), you will see the first 13 points are generated in 4 calls, as the generator has been changed. One things bothers me though: at initialization, I think I could generate fewer points (I generate 13 on one side and then a few additional on the other side to get a balanced interval). This happens only at start, and in many cases once started time evolves always in the same direction, so it may not be a major problem. You have to add some print statements or run under a debugger to see how the calls to generate are performed.


No, the Frame instances would still be built as they are now so they can
be linked together with a simple tree. The frames would contain a cache
for the transforms, which would implement TimeStamped.

yes, this makes more sense, I guess my thinking was to complicated.

I'm not sure yet if we should set up interpolation between transforms at
frame level and removing the specialized cache for lower level
computations like precession-nutation model, or if we should preserve
both levels (just like processors have level 1 and level 2 caches, and
sometimes even level 3).

I have no idea about that right now, but it would be good to have an
environment for performance testing different settings for different
scenarios.

What do you think about the way multithreading is handled with
ReentrantReadWriteLock at slots end entries level ? Does it seem
appropriate to you ? Did you find any design or implementation problem
on this specific topic ?

At first I was a bit musing about the slot mechanism, but when I
understand it correctly, your intention was to have fine grained control
over locking and eviction.

Yes, and I wanted to make sure the feature would scale well for a large number of requests at random times.

I have also looked into the multi-threading
part, but did not find a problem yet.

One important thing is that due to the current design of the cache, the
generator must itself be thread-safe, as it can happen that multiple
threads request entries from the generator. The javadoc states the
opposite, which is wrong imho.

Then I did something wrong because what the javadoc says is really what I wanted to have. The generator should be called only when the write lock of the slot is owned by the thread that calls it. Perhaps it should alwo own the write lock of the complete cache ?

Thanks for your review
Luc


Thomas





----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.