I post from a messages on harbour ng posted by Przemek
here you can follow tread
http://n2.nabble.com/XBASE%2B%2B-speedtst-td2545191.html
here you download spedtst.prg
http://harbour-project.svn.sourceforge.net/viewvc/harbour-project/trunk/harbour/tests/speedtst.prg?view=markup


Looks that xbase++ is not MT safe in some basic tests.
I have information that in randomly GPFs in T023 or T044 on
different computers. You can try to exploit it by:
  speedtst --thread=2 --scale --only=023,044
Please try to repeat above tests few time on different machines.
It's also possible that the problem is not exactly in this tests
but somewhere else anyhow definitely there is sth wrong.

It's a very serious problem which should be reported to xbase++ authors.
I wanted to test the automatic item write protection in xbase++
in some unpleasure to protect situations but looks that it fails
internally in places which should not cause any problems so any
more advanced tests with this language does not make sense until
basic MT functionality will not be fixed.

It also causes that it's hard to say something about xbase++ MT
solutions. I do not know xbase++ code and I can only guess some
things using information from tests and some necessary and enough
conditions which have to be passed to make MT programs safe.
But when xabse++ crashes like above then it means that some fundamental
conditions I'm using are false.

I think that xbase++ users should report this problem to some
xbase++ support list with speedtst code as example.

best regards,
Przemek

hi,


did you have "test" it ?

> I have information that in randomly GPFs in T023 or T044 on

AFaik it work with v331 + Hotfix AND v350

greetings by OHR
Jimmy

Przemek, Massimo, Angel,

>Looks that xbase++ is not MT safe in some basic tests.

I'm not sure how you come to this conclusion, but:

>It GPF'd with this log:

>FATAL ERROR LOG
>Not recoverable Error!
>SYS Thread-ID: 1224
>Module: MOM
>Error Codes: EH: 1006 Sub: 0(0) OS: 0 XPP: 15
>Call Stack of Thread 1 (1732):
>@NOTIFIER@I@SUBSCRIBE(0)
>HB_MUTEXSUBSCRIBE(0)
>TEST(0)
>MAIN(0)
>Call Stack of Thread 2 (1548):
>Call Stack of Thread 3 (1224):
>T044(0)
>THTESTSCALE(0)
>Call Stack of Thread 4 (1180):
>T044(0)
>THTESTSCALE(0)
>File: C:\experimentos\speedhb\speedxpp.exe
>TimeStamp: 20090327 15:52
>End of FATAL ERROR LOG.

This is not a GPF, but the program simply ran out of Memory, because you 
were creating 2000000 or so MOM objects -- in form of (empty) arrays -- 
in a loop without any other code. If you put some sleep() or some other 
code in this loop, so that the GC has time to kick in, then you wouldn't 
get this runtime error. Anyway, its not a GPF and it has absolutely 
nothing to do with multi-threading.

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas@Aerospace-History.net
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Przemek, Massimo, Angel,

A simple proof for this, is to increase the loops to 5 Million (from 
1 Million), and run it with a single thread. You will get the same 
out-of-memory error.

Also, if you reduce the number of loops to 500,000, you can run two or
even three threads without any error.

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas@Aerospace-History.net
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Hello Andreas Gehrs-Pahl,

> A simple proof for this, is to increase the loops to 5 Million (from 1
> Million), and run it with a single thread. You will get the same
> out-of-memory error.
> 
> Also, if you reduce the number of loops to 500,000, you can run two or
> even three threads without any error.

However, it might be considered a minor Xbase++ defect to not perform a GC 
cycle when approaching memory exhaustion.  The GC should not need to rely 
on developer inserted code sequences (sleeps) to prevent memory exhaustion 
during long, non-interactive thread executions.  

Nevertheless, it is not a bad idea to allow long running (MT) processes to 
take a breather once in a while since Xbase++ is still bound to a single 
CPU which may block lower priority threads from adequate CPU time.

Of course this does not remove the 2M MOM limit in the current Xbase++ implementation 
which is not a defect, but rather an implementation decision.  IMO, 2M is 
appropriate for the 800M memory space for Xbase++ variables which allows 
an approx average 4K per MOM object.  This should be raised if a 64bit version 
of Xbase++ is ever released.

Regards,

Rodd Graham, Consultant
Graham Automation Systems, LLC

Rodd,

>However, it might be considered a minor Xbase++ defect to not perform a GC 
>cycle when approaching memory exhaustion. The GC should not need to rely 
>on developer inserted code sequences (sleeps) to prevent memory exhaustion 
>during long, non-interactive thread executions.

I agree that this XppFatal error due to memory exhaustion should certainly 
be considered a defect, and I hope it will be fixed at some time, even 
though I won't hold my breath. 

>Of course this does not remove the 2M MOM limit in the current Xbase++ 
>implementation which is not a defect, but rather an implementation decision.
>IMO, 2M is appropriate for the 800M memory space for Xbase++ variables which 
>allows an approx average 4K per MOM object. This should be raised if a 64bit 
>version of Xbase++ is ever released.

Is this 2M MOM limit documented anywhere? I think that Xbase++ should be
able to handle more than 800M of memory, anyway, IMNSHO.

The following, condensed, version of the program will create the same fatal 
error, but it doesn't reveal any "very serious problem" with multi-threading
in Xbase++, as the original poster alleged!

Procedure Main()
LOCAL aArray, nCount, nSeconds := Seconds()
   for nCount := 1 to 2500000
       aArray := {}
   next
   QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds)))
return

The following, slightly modified program will run without an error, though, 
as it is about 50% slower, which gives the GC enough time to clean up, at 
least on my computer!

Procedure Main()
LOCAL aArray, nCount, nSeconds := Seconds()
   for nCount := 1 to 2500000
       aArray := Array(0)
   next
   QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds)))
return

A PDR or a note in the documentation would be nice, though.

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas@Aerospace-History.net
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Hello Andreas Gehrs-Pahl,

>> Of course this does not remove the 2M MOM limit in the current
>> Xbase++ implementation which is not a defect, but rather an
>> implementation decision. IMO, 2M is appropriate for the 800M memory
>> space for Xbase++ variables which allows an approx average 4K per MOM
>> object. This should be raised if a 64bit version of Xbase++ is ever
>> released.
>> 
> Is this 2M MOM limit documented anywhere? I think that Xbase++ should
> be able to handle more than 800M of memory, anyway, IMNSHO.

It is not documented except in historical NG posts and communications I have 
had with Alaska support.  My personal experience is that I rarely exceed 
250K MOM objects which still has plenty of breathing room.  

The 800M limit is a result of Xbase++ partitioning the 2GB address space 
of a 32 bit process at startup.  It is a design decision of Alaska Software 
that makes sense in light of the fact that Xbase++ uses its own heap manager 
that seems to be written for a single contiguous block of memory AND 3rd 
party in-process DLLs need memory space for their operation outside of the 
Xbase++ heap.

> The following, condensed, version of the program will create the same
> fatal error, but it doesn't reveal any "very serious problem" with
> multi-threading in Xbase++, as the original poster alleged!

FWIW, the original poster (Przemek) is a highly skilled C developer on the 
harbour project who has been implementing MT in harbour using Xbase++ as 
a model for compatibility.  He is the harbour counterpart to Steffen of which 
I respect both of their skill sets.  However, Przemek is not an Xbase++ expert/enthusiast 
as he is committed to open source software.

> 
> Procedure Main()
> LOCAL aArray, nCount, nSeconds := Seconds()
> for nCount := 1 to 2500000
> aArray := {}
> next
> QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds)))
> return
> The following, slightly modified program will run without an error,
> though, as it is about 50% slower, which gives the GC enough time to
> clean up, at least on my computer!
> 
> Procedure Main()
> LOCAL aArray, nCount, nSeconds := Seconds()
> for nCount := 1 to 2500000
> aArray := Array(0)
> next
> QOut('Seconds: ' + alltrim(str(Seconds() - nSeconds)))
> return
> A PDR or a note in the documentation would be nice, though.
> 

It would be interesting to know if it is the Array() call processing overhead 
(slow down), the Array() call (entry/exit code), or the actual Array() implementation 
that allows/causes the GC to catch up.

Regards,

Rodd Graham, Consultant
Graham Automation Systems, LLC

Followup Test...

Rodd,

For some reason, my respond to your post is classified as Spam by the 
Alaska News Server and rejected. I don't know why exactly, but probably
because it contains some words or word combinations that Alaska doesn't
like. Anyway, I have never seen this before, and I'm quite surprised.

I would really like to know what words Alaska considers so offending 
that they rejected this post.

Anyway, I was just pointing out that I was referring to the "original 
poster" in this NG and the one that called it a "GPF" on that other
site. Interestingly, neither one felt the need to correct their 
misconception or cross-post there.

Maybe this post will go through, as I omitted all names, and didn't
mention that the test isn't applicable to Xbase++ anyway, due to the
single processor proc-affinity requirement of Xbase++.

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas@Aerospace-History.net
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Hi,

I have testet the code on 2 PCs:

[b]No Errors here ![/b]

PCs:
PC1: AMD Athlon 64 3000++ (1800 Mhz), 1GB Ram, Win2000 SP4
PC2: AMD Athlon X2 4200++ (2200 Mhz), 2GB Ram, WinXP SP3
Compiler:
XPP 1.82.294
XPP 1.90.331
XPP 1.90.350

The results are very close together, but not in this test:

PC1-1.82.294:
   [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]...39.84
PC1-1.90.331:
   [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.43
PC1-1.90.350:
   [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.53
      Beta

PC2-1.82.294:
   [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]...66.73
PC2-1.90.331:
   [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.11
PC2-1.90.350:
   [ T042: iif( i%1000==0, a:={}, ), aadd(a,{i,1,.t.,s,s2,a2, ]....6.13
      Beta

I think the SL1-RC1 will be a little faster after Release,
but maybe the little difference is caused by other tasks.
I did not change the 1-CPU setting, just compiled it as it is.

I think that the speed increase is caused of the internal use of LONG
vars if a integer value is detected. But I can't know it.

There was no problem to run the test here !

Bye
Hubert

Here are the result files:

http://www.xbaseforum.de/viewtopic.php?f=32&t=3187&p=33219#p33219

----------------

Ich empfehle:  www.xbaseforum.de  (in deutsch)

Homepage:

German  - www.familie-brandel.de/index.htm
English - www.familie-brandel.de/index_e.htm

Hubert,


> I think that the speed increase is caused of the internal use of LONG
> vars if a integer value is detected. But I can't know it.

on 32-bit machines, integer and long values are both dwords.  Or did you
mean something else?

Regards,

Phil Ide

---------------------
www.xbhcl.com
www.pbih.eu
www.idep.nl
---------------------
I am not part of the problem. I am a Republican. - Dan Quayle

Phil Ide schrieb:
> Or did you mean something else?

Hi,

I mean LONG instead of FLOAT (or what the normal type from Xbase++ is 
called) ...

Bye
Hubert

----------------

Ich empfehle:  www.xbaseforum.de  (in deutsch)

Homepage:

German  - www.familie-brandel.de/index.htm
English - www.familie-brandel.de/index_e.htm

Hello Phil:
good morning.
We all was missing your presence. Are you back again? Where where you?

Beling
Best regards

Phil Ide escreveu:
> Hubert,
> 
> 
>> I think that the speed increase is caused of the internal use of LONG
>> vars if a integer value is detected. But I can't know it.
> 
> on 32-bit machines, integer and long values are both dwords.  Or did you
> mean something else?
> 
> Regards,

Carlos,

> Hello Phil:
> good morning.
> We all was missing your presence. Are you back again? Where where you?

Ye, I'm back again.  I'm currently at spending my days at home enjoying the
recession.  I had to move home again last year, but hopefully I'll be able
to lay some roots this time around.

Regards,

Phil Ide

---------------------
www.xbhcl.com
www.pbih.eu
www.idep.nl
---------------------
Every day is starting to look like Monday ...

Hello Phil:
welcome.

Beling
Best regards

Phil Ide escreveu:
> Carlos,
> 
>> Hello Phil:
>> good morning.
>> We all was missing your presence. Are you back again? Where where you?
> 
> Ye, I'm back again.  I'm currently at spending my days at home enjoying the
> recession.  I had to move home again last year, but hopefully I'll be able
> to lay some roots this time around.
> 
> Regards,