Author | Topic: A very serious problem which should be reported to xbase++ authors | |
---|---|---|
Massimo | A very serious problem which should be reported to xbase++ authors on Sat, 28 Mar 2009 16:04:02 +0100 I post from a messages on harbour ng posted by Przemek here you can follow tread http://n2.nabble.com/XBASE%2B%2B-speedtst-td2545191.html here you download spedtst.prg http://harbour-project.svn.sourceforge.net/viewvc/harbour-project/trunk/harbour/tests/speedtst.prg?view=markup Looks that xbase++ is not MT safe in some basic tests. I have information that in randomly GPFs in T023 or T044 on different computers. You can try to exploit it by: speedtst --thread=2 --scale --only=023,044 Please try to repeat above tests few time on different machines. It's also possible that the problem is not exactly in this tests but somewhere else anyhow definitely there is sth wrong. It's a very serious problem which should be reported to xbase++ authors. I wanted to test the automatic item write protection in xbase++ in some unpleasure to protect situations but looks that it fails internally in places which should not cause any problems so any more advanced tests with this language does not make sense until basic MT functionality will not be fixed. It also causes that it's hard to say something about xbase++ MT solutions. I do not know xbase++ code and I can only guess some things using information from tests and some necessary and enough conditions which have to be passed to make MT programs safe. But when xabse++ crashes like above then it means that some fundamental conditions I'm using are false. I think that xbase++ users should report this problem to some xbase++ support list with speedtst code as example. best regards, Przemek | |
AUGE_OHR | Re: A very serious problem which should be reported to xbase++ authors on Sat, 28 Mar 2009 19:20:46 +0100 hi, did you have "test" it ? > I have information that in randomly GPFs in T023 or T044 on AFaik it work with v331 + Hotfix AND v350 greetings by OHR Jimmy | |
Andreas Gehrs-Pahl | Re: A very serious problem which should be reported to xbase++ authors on Sat, 28 Mar 2009 14:32:28 -0400 Przemek, Massimo, Angel, >Looks that xbase++ is not MT safe in some basic tests. I'm not sure how you come to this conclusion, but: >It GPF'd with this log: >FATAL ERROR LOG >Not recoverable Error! >SYS Thread-ID: 1224 >Module: MOM >Error Codes: EH: 1006 Sub: 0(0) OS: 0 XPP: 15 >Call Stack of Thread 1 (1732): >@NOTIFIER@I@SUBSCRIBE(0) >HB_MUTEXSUBSCRIBE(0) >TEST(0) >MAIN(0) >Call Stack of Thread 2 (1548): >Call Stack of Thread 3 (1224): >T044(0) >THTESTSCALE(0) >Call Stack of Thread 4 (1180): >T044(0) >THTESTSCALE(0) >File: C:\experimentos\speedhb\speedxpp.exe >TimeStamp: 20090327 15:52 >End of FATAL ERROR LOG. This is not a GPF, but the program simply ran out of Memory, because you were creating 2000000 or so MOM objects -- in form of (empty) arrays -- in a loop without any other code. If you put some sleep() or some other code in this loop, so that the GC has time to kick in, then you wouldn't get this runtime error. Anyway, its not a GPF and it has absolutely nothing to do with multi-threading. -- Andreas --- --- Andreas Gehrs-Pahl E-Mail: GPahl@CharterMI.net 415 Gute Street or: Andreas@DDPSoftware.com Owosso, MI 48867-4410 or: Andreas@Aerospace-History.net Tel: (989) 723-9927 Web Site: http://www.Aerospace-History.net --- --- | |
Andreas Gehrs-Pahl | Re: A very serious problem which should be reported to xbase++ authors on Sat, 28 Mar 2009 16:00:04 -0400 Przemek, Massimo, Angel, A simple proof for this, is to increase the loops to 5 Million (from 1 Million), and run it with a single thread. You will get the same out-of-memory error. Also, if you reduce the number of loops to 500,000, you can run two or even three threads without any error. -- Andreas --- --- Andreas Gehrs-Pahl E-Mail: GPahl@CharterMI.net 415 Gute Street or: Andreas@DDPSoftware.com Owosso, MI 48867-4410 or: Andreas@Aerospace-History.net Tel: (989) 723-9927 Web Site: http://www.Aerospace-History.net --- --- | |
Rodd Graham | Re: A very serious problem which should be reported to xbase++ authors on Mon, 30 Mar 2009 06:44:22 +0000 Hello Andreas Gehrs-Pahl, > A simple proof for this, is to increase the loops to 5 Million (from 1 > Million), and run it with a single thread. You will get the same > out-of-memory error. > > Also, if you reduce the number of loops to 500,000, you can run two or > even three threads without any error. However, it might be considered a minor Xbase++ defect to not perform a GC cycle when approaching memory exhaustion. The GC should not need to rely on developer inserted code sequences (sleeps) to prevent memory exhaustion during long, non-interactive thread executions. Nevertheless, it is not a bad idea to allow long running (MT) processes to take a breather once in a while since Xbase++ is still bound to a single CPU which may block lower priority threads from adequate CPU time. Of course this does not remove the 2M MOM limit in the current Xbase++ implementation which is not a defect, but rather an implementation decision. IMO, 2M is appropriate for the 800M memory space for Xbase++ variables which allows an approx average 4K per MOM object. This should be raised if a 64bit version of Xbase++ is ever released. Regards, Rodd Graham, Consultant Graham Automation Systems, LLC | |
Andreas Gehrs-Pahl | Re: A very serious problem which should be reported to xbase++ authors on Tue, 31 Mar 2009 02:55:28 -0400 Rodd, >However, it might be considered a minor Xbase++ defect to not perform a GC >cycle when approaching memory exhaustion. The GC should not need to rely >on developer inserted code sequences (sleeps) to prevent memory exhaustion >during long, non-interactive thread executions. I agree that this XppFatal error due to memory exhaustion should certainly be considered a defect, and I hope it will be fixed at some time, even though I won't hold my breath. >Of course this does not remove the 2M MOM limit in the current Xbase++ >implementation which is not a defect, but rather an implementation decision. >IMO, 2M is appropriate for the 800M memory space for Xbase++ variables which >allows an approx average 4K per MOM object. This should be raised if a 64bit >version of Xbase++ is ever released. Is this 2M MOM limit documented anywhere? I think that Xbase++ should be able to handle more than 800M of memory, anyway, IMNSHO. The following, condensed, version of the program will create the same fatal error, but it doesn't reveal any "very serious problem" with multi-threading in Xbase++, as the original poster alleged! Procedure Main() LOCAL aArray, nCount, nSeconds := Seconds() for nCount := 1 to 2500000 aArray := {} next QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds))) return The following, slightly modified program will run without an error, though, as it is about 50% slower, which gives the GC enough time to clean up, at least on my computer! Procedure Main() LOCAL aArray, nCount, nSeconds := Seconds() for nCount := 1 to 2500000 aArray := Array(0) next QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds))) return A PDR or a note in the documentation would be nice, though. -- Andreas --- --- Andreas Gehrs-Pahl E-Mail: GPahl@CharterMI.net 415 Gute Street or: Andreas@DDPSoftware.com Owosso, MI 48867-4410 or: Andreas@Aerospace-History.net Tel: (989) 723-9927 Web Site: http://www.Aerospace-History.net --- --- | |
Rodd Graham | Re: A very serious problem which should be reported to xbase++ authors on Tue, 31 Mar 2009 10:33:37 +0000 Hello Andreas Gehrs-Pahl, >> Of course this does not remove the 2M MOM limit in the current >> Xbase++ implementation which is not a defect, but rather an >> implementation decision. IMO, 2M is appropriate for the 800M memory >> space for Xbase++ variables which allows an approx average 4K per MOM >> object. This should be raised if a 64bit version of Xbase++ is ever >> released. >> > Is this 2M MOM limit documented anywhere? I think that Xbase++ should > be able to handle more than 800M of memory, anyway, IMNSHO. It is not documented except in historical NG posts and communications I have had with Alaska support. My personal experience is that I rarely exceed 250K MOM objects which still has plenty of breathing room. The 800M limit is a result of Xbase++ partitioning the 2GB address space of a 32 bit process at startup. It is a design decision of Alaska Software that makes sense in light of the fact that Xbase++ uses its own heap manager that seems to be written for a single contiguous block of memory AND 3rd party in-process DLLs need memory space for their operation outside of the Xbase++ heap. > The following, condensed, version of the program will create the same > fatal error, but it doesn't reveal any "very serious problem" with > multi-threading in Xbase++, as the original poster alleged! FWIW, the original poster (Przemek) is a highly skilled C developer on the harbour project who has been implementing MT in harbour using Xbase++ as a model for compatibility. He is the harbour counterpart to Steffen of which I respect both of their skill sets. However, Przemek is not an Xbase++ expert/enthusiast as he is committed to open source software. > > Procedure Main() > LOCAL aArray, nCount, nSeconds := Seconds() > for nCount := 1 to 2500000 > aArray := {} > next > QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds))) > return > The following, slightly modified program will run without an error, > though, as it is about 50% slower, which gives the GC enough time to > clean up, at least on my computer! > > Procedure Main() > LOCAL aArray, nCount, nSeconds := Seconds() > for nCount := 1 to 2500000 > aArray := Array(0) > next > QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds))) > return > A PDR or a note in the documentation would be nice, though. > It would be interesting to know if it is the Array() call processing overhead (slow down), the Array() call (entry/exit code), or the actual Array() implementation that allows/causes the GC to catch up. Regards, Rodd Graham, Consultant Graham Automation Systems, LLC | |
Andreas Gehrs-Pahl | Re: A very serious problem which should be reported to xbase++ authors on Tue, 31 Mar 2009 20:07:15 -0400 Followup Test... | |
Andreas Gehrs-Pahl | Re: A very serious problem which should be reported to xbase++ authors on Tue, 31 Mar 2009 20:30:41 -0400 Rodd, For some reason, my respond to your post is classified as Spam by the Alaska News Server and rejected. I don't know why exactly, but probably because it contains some words or word combinations that Alaska doesn't like. Anyway, I have never seen this before, and I'm quite surprised. I would really like to know what words Alaska considers so offending that they rejected this post. Anyway, I was just pointing out that I was referring to the "original poster" in this NG and the one that called it a "GPF" on that other site. Interestingly, neither one felt the need to correct their misconception or cross-post there. Maybe this post will go through, as I omitted all names, and didn't mention that the test isn't applicable to Xbase++ anyway, due to the single processor proc-affinity requirement of Xbase++. -- Andreas --- --- Andreas Gehrs-Pahl E-Mail: GPahl@CharterMI.net 415 Gute Street or: Andreas@DDPSoftware.com Owosso, MI 48867-4410 or: Andreas@Aerospace-History.net Tel: (989) 723-9927 Web Site: http://www.Aerospace-History.net --- --- | |
Hubert Brandel | Re: A very serious problem which should be reported to xbase++ authors on Sat, 28 Mar 2009 20:08:28 +0100 Hi, I have testet the code on 2 PCs: [b]No Errors here ![/b] PCs: PC1: AMD Athlon 64 3000++ (1800 Mhz), 1GB Ram, Win2000 SP4 PC2: AMD Athlon X2 4200++ (2200 Mhz), 2GB Ram, WinXP SP3 Compiler: XPP 1.82.294 XPP 1.90.331 XPP 1.90.350 The results are very close together, but not in this test: PC1-1.82.294: [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]...39.84 PC1-1.90.331: [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.43 PC1-1.90.350: [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.53 Beta PC2-1.82.294: [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]...66.73 PC2-1.90.331: [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.11 PC2-1.90.350: [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.13 Beta I think the SL1-RC1 will be a little faster after Release, but maybe the little difference is caused by other tasks. I did not change the 1-CPU setting, just compiled it as it is. I think that the speed increase is caused of the internal use of LONG vars if a integer value is detected. But I can't know it. There was no problem to run the test here ! Bye Hubert Here are the result files: http://www.xbaseforum.de/viewtopic.php?f=32&t=3187&p=33219#p33219 ---------------- Ich empfehle: www.xbaseforum.de (in deutsch) Homepage: German - www.familie-brandel.de/index.htm English - www.familie-brandel.de/index_e.htm | |
Phil Ide | Re: A very serious problem which should be reported to xbase++ authors on Tue, 31 Mar 2009 01:46:18 +0100 Hubert, > I think that the speed increase is caused of the internal use of LONG > vars if a integer value is detected. But I can't know it. on 32-bit machines, integer and long values are both dwords. Or did you mean something else? Regards, Phil Ide --------------------- www.xbhcl.com www.pbih.eu www.idep.nl --------------------- I am not part of the problem. I am a Republican. - Dan Quayle | |
Hubert Brandel | Re: A very serious problem which should be reported to xbase++ authors on Tue, 31 Mar 2009 07:17:44 +0200 Phil Ide schrieb: > Or did you mean something else? Hi, I mean LONG instead of FLOAT (or what the normal type from Xbase++ is called) ... Bye Hubert ---------------- Ich empfehle: www.xbaseforum.de (in deutsch) Homepage: German - www.familie-brandel.de/index.htm English - www.familie-brandel.de/index_e.htm | |
Carlos Beling | Re: A very serious problem which should be reported to xbase++ authors on Tue, 31 Mar 2009 10:19:49 -0300 Hello Phil: good morning. We all was missing your presence. Are you back again? Where where you? Beling Best regards Phil Ide escreveu: > Hubert, > > >> I think that the speed increase is caused of the internal use of LONG >> vars if a integer value is detected. But I can't know it. > > on 32-bit machines, integer and long values are both dwords. Or did you > mean something else? > > Regards, | |
Phil Ide | Re: A very serious problem which should be reported to xbase++ authors on Wed, 01 Apr 2009 00:09:57 +0100 Carlos, > Hello Phil: > good morning. > We all was missing your presence. Are you back again? Where where you? Ye, I'm back again. I'm currently at spending my days at home enjoying the recession. I had to move home again last year, but hopefully I'll be able to lay some roots this time around. Regards, Phil Ide --------------------- www.xbhcl.com www.pbih.eu www.idep.nl --------------------- Every day is starting to look like Monday ... | |
Carlos Beling | Re: A very serious problem which should be reported to xbase++ authors on Wed, 01 Apr 2009 10:03:12 -0300 Hello Phil: welcome. Beling Best regards Phil Ide escreveu: > Carlos, > >> Hello Phil: >> good morning. >> We all was missing your presence. Are you back again? Where where you? > > Ye, I'm back again. I'm currently at spending my days at home enjoying the > recession. I had to move home again last year, but hopefully I'll be able > to lay some roots this time around. > > Regards, |