OpenEuphoria

2. Re: Memory leak?

• Posted by mattlewis (admin) Feb 03, 2014
• 3849 views

That's an old version. The latest that I posted to the archive is 1.34, though the source code itself says 1.36. It looks like 1.33 got several updates, including some memory leak fixes:

You can download the updated file here: https://sourceforge.net/p/eudbc/code/HEAD/tree/trunk/odbc.e

Matt

4. Re: Memory leak?

• Posted by gwalters Feb 04, 2014
• 3713 views

Matt,

I tried the newer verson of odbc.e and there was some improvement. Using the program with the same data the old version memory grew to 333Mb and the newer version grew to 304Mb. Got this by looking at task manager while the program ran (about 3 minutes). The memory required continues to grow until either the program terminates normally or runs out of memory.

The program is looking at a 2 degree square window of stars (Tycho satellite data) looking for colinear triplets. There are 112 stars in the window and it recursively creates a cursor of 111 stars 112 times searching for star triplets. The window cannot be larger because the program will give out of memory eventually.

So, there is some improvement but the basic problem is still present. Thanks for checking.

6. Re: Memory leak?

• Posted by gwalters Feb 12, 2014
• 3682 views

8. Re: Memory leak?

• Posted by DerekParnell (admin) Feb 12, 2014
• 3580 views

If you are taking 112 stars at a time, there are only 227920 combinations to check. You could calculate (once) a list of all those combinations

{ 
{ 1, 2, 3}, 
{ 1, 2, 4}, 
{ 1, 2, 5}, ... 
{ 110, 111, 112} 
} 

This would take about 4.5MB. Then grab the 'next' 112 stars, assigning each a number 1 to 112. Then loop through the list of combinations, checking each co-linear aspect of the group of three, and only save or report on ones that are within tolerance. You do not have to have all the stars in RAM at the same time, in fact you would only need three stars at a time. Just grab the details of the next 'three' off disk.

10. Re: Memory leak?

• Posted by gimlet Feb 13, 2014
• 3602 views

Really!

OK, so we make a list of 112 stars with their (x,y) coordinates. - how accurate?? We can have 111 stars as their first match, 110 stars as their second match = 112 * 111 * 110 = rather a lot more than 227,000 (where did that number come from?).

Second: If the distance between s1 and s2 is small (ie very small in relation to the distance from the origin) how accurate is any measurement going to be?

Third: how do you determine 3 points are collinear? That is given 3 points (x1,y1).. how will you determine they are collinear and not run into problems with dividing values by other values close to zero.

12. Re: Memory leak?

• Posted by gimlet Feb 13, 2014
• 3532 views

Jim

I choose 1 of 112 = first star I choose 1 of 111 = second star There are 110 stars to check for collinearity

112!/3! OK

Second:

Checking for collinearity is not fall over simple. See for example:

Lecture Notes on Geometric Robustness by JR Shewchuk

14. Re: Memory leak?

• Posted by gimlet Feb 13, 2014
• 3515 views

Jim

The 112!/3! was a stupid error and certainly not = 112*111*110. I realised after I submitted that it was stupid.

I choose 3 stars in order. If I choose 1 2 3 then I will not choose 2 1 3 later and vice versa. But I could have as I did not specify a particular order (like the key of a combination lock).

So, I guess, Derek is wrong on that.

Second the point about accuracy. Quoting:

"I'm looking for star triplets that are in a straight line and are not off the straight line more then 1 arc sec."

That sounds to me like testing for collinearity with stars close together will have accuracy problems.

Third:

I would have thought Derek's approach was obvious but it has a problem in that a star may be collinear with a star or stars not in the 112 chosen. I have no idea how close together stars need to be to be considered "in a line". This is a complication.

16. Re: Memory leak?

• Posted by jimcbrown (admin) Feb 13, 2014
• 3508 views

Okay.

Ok, so it's a combination without repetition in that case.

If you did so, then you'd change it into a permutation. Why would gwalters want to do that though? He's better off doing a combination if he can get away with it (as that means fewer overall computations).

I disagree. If we both agree that it's a case of using combination without repetition, then Derek's original number is correct.

If gwalters is doing a permutation instead (either because he has to, or perhaps because gwalters enjoys the music of gnashing hard disks and the flash of blinkenlights), then Derek's number is wrong.

The data on star position is supposedly accurate to 2.5 milliarcseconds. So the margin of error is 400 times smaller than what gwalters is looking for. There's always room for errors in measurement and other forms of accuracy, but it seems that gwalters has plently of wiggle room.

I agreed that this looks like a potential complication, but it's also one present in the original EDBC using program.

18. Re: Memory leak?

• Posted by DerekParnell (admin) Feb 13, 2014
• 3464 views

Huh? I assumed that checking stars 1, 2, 3 would be identical to checking stars 2, 1, 3. Am I wrong there?

Yep ... that would be an issue. The '112' might have to be flexible. One could sort all the stars such that they are grouped into approximate or near co-linear and then apply a 'sliding' window over them. Not a trivial exercise though.

20. Re: Memory leak?

• Posted by gwalters Feb 14, 2014
• 3411 views

As far as the accuracy of tyco-2000. The above quoted 2.5 mili-arc-seconds/year is for the proper motion of a star, it's motion across the sky relative to the fixed background of stars. The positional accuracy is 60 mas. All this is per tyco literature.

What I have found so far, looking at 60min windows, is that 10-30 persent of the stars ar colinear to less thatn 1 arcsec at least for the areas I've looked at. Program blows up if I look at a window larger or there are too may stars in the window. I now have to fugure out why. That's the real problem.

Thanks for all your interest and ideas on how to improve the program. I obviously need to improve the algorithm.

22. Re: Memory leak?

• Posted by gimlet Feb 15, 2014
• 3324 views

The way you say this seems to imply the stars are well separated.

If they are not the procedure could be unreliable.

Consider A is distant from B but C is close.

You would (in this case) measure the distance A is from the line BC.

Another way your code may be unreliable is that you use tan, sin and division with numbers which can be close to or equal to zero. Double precision is very accurate but it looks as though some the numbers you could encounter could differ by a factor as large as 10^12.

E.G. line

tanPhi = (star2[DECnumeric] - star1[DECnumeric])/(star2[RAnumeric] - star1[RAnumeric]) 

What if star2[RAnumeric] = star1[RAnumeric]? or

distance = ((star3[DECnumeric]- star1Rec[DECnumeric]) / tanPhi 
	 - (star3[RAnumeric]- star1Rec[RAnumeric])) * sinPhi 

Both tanPhi and sinPhi can approach zero and tanPhi can approach +-inf.

Perhaps you could do something this:

Move the origin to the central star's position. (get distances relative to B) 
Determine the closer of the outer stars (A). 
Determine the the angle BAC etc., 

The book Lecture Notes on Geometric Robustness by JR Shewchuk is available free on the web. And it seems rather good.

24. OT: Colinearity of Stars in Our Galaxy

• Posted by SDPringle Feb 15, 2014
• 3311 views

G. Walters,

Somethings that mean seem impossibly unlikely may be actually probable. First thing I would ask is what is the density of stars in the data you are looking at. How many stars are in your 60min by 60min Windows? Perhaps it is just a statistical property of having that many dots in a 3600x3600 grid, generally means that 10-30 % are co-linear. The fact that the galaxy is in a flat shape like a plate makes the stars in space unlike random dots in a grid. You are looking at the plate from its edge, after all.

Play pool, often I cannot find the cue ball, another and a hole are co-linear at the end of the game when there are fewer balls. It is much easier at the beginning of the game. The question here is what is the probability of getting up to 30% of the points colinear in such a grid given this bias of the galaxy is deformed into a plate and our perspective?

S D Pringle

26. Re: OT: Colinearity of Stars in Our Galaxy

• Posted by gwalters Feb 15, 2014
• 3325 views

[Edit: Fixed the quote box above here: S D Pringle]

You have got to the heart of the problem. At first I thought that the 2 outer stars were orbiting each other around the center of mass and the central star was resting at a neutral point between. The results seen, looking at the proper motion, killed that idea. So I am left with the problem of how/why are they aligned so closely. The regions I've looked at are not through the milky way, but mostly perpendicular. (i.e. 5,0,0 RA and 5,0,0 DEC)

28. Re: OT: Colinearity of Stars in Our Galaxy

• Posted by gimlet Feb 15, 2014
• 3288 views

Perhaps what you are seeing is a star stream. I would have thought that the stars were mostly independent (you say the triples are well separated). Do they have similar alignments? One would expect galaxy stars would be (more or less) randomly aligned. A star stream on the other hand would not be randomly aligned (common direction).

The other day I was at the Powerhouse and one of the exhibits was iron filings in a magnetic field. The pictures you usually see are nicely aligned to the lines of force - great visualisations. What I saw was something only a bit like that: a large clump of filings which tended to stick together not very well aligned in the small but in the large obviously constrained by the field.