Hi all,

Did anyone have the experience where a DBF grows to 4 GB?
I am running my apps on a Windows Server 2008 with about 20 users where
this now happened for the 3rd time on two different databases. The
databases are completely corrupt and I either had to resolve to backups or
salvage the data in the hard way by editing the database in text format.

Any ideas why this happened please.

Ben

hi,

> Did anyone have the experience where a DBF grows to 4 GB?

did you use DBFNTX ... with Cl*pper ?

greetings by OHR
Jimmy

No, I am usimg DBFCDX. Clipper plays no role here.

Ben

4 gig is max for a 32 bit address space.

to get above that you need a SQL backend.

Jim



On 05/24/2012 06:11 PM, Ben Bodenstein wrote:
>
> No, I am usimg DBFCDX. Clipper plays no role here.
>
> Ben

Am 25.05.2012 04:22, schrieb James Loughner:
> 4 gig is max for a 32 bit address space.
>
> to get above that you need a SQL backend.
>
> Jim
>
>
>
> On 05/24/2012 06:11 PM, Ben Bodenstein wrote:
>>
>> No, I am usimg DBFCDX. Clipper plays no role here.
>>
>> Ben
>
I think his problem is, that the DBF should NOT be so big and corruped 

Maybe this is a problem with the metadata on a 2008 Server, there was a 
posting how to avoid those problems.

Bye
Hubert

James Loughner wrote:

> 4 gig is max for a 32 bit address space.
> 
> to get above that you need a SQL backend.

I would say that he does not actually have that much data - but the DBF
file suddenly has this size without any obvious reason.

Thomas

You can use ADS, they solved this problem
> No, I am usimg DBFCDX. Clipper plays no role here.
>
> Ben

Best regards, Altiy

Ben Bodenstein wrote:

> Did anyone have the experience where a DBF grows to 4 GB?
> I am running my apps on a Windows Server 2008

Do you have this problem in multiple places? Or always on the same physical
machine?

> Any ideas why this happened please.

File system problems or even low level disk hardware problems maybe.

Trouble is that we know nothing about the database and how it is used. 
Could be that it needs to be regularly packed or recycle deleted 
records. But the fact is 4 gig is max file size. Without some detail it 
is difficult to say why.

Jim



On 05/25/2012 08:16 AM, Thomas Braun wrote:
> Ben Bodenstein wrote:
>
>> Did anyone have the experience where a DBF grows to 4 GB?
>> I am running my apps on a Windows Server 2008
>
> Do you have this problem in multiple places? Or always on the same physical
> machine?
>
>> Any ideas why this happened please.
>
> File system problems or even low level disk hardware problems maybe.

Hi all,

Thanks for all the replies. Maybe I didn't make myself clear, it is not the
data that is getting to 4 GB, the data is only a few 100 kb's.
Something causes the database to grow to 4 GB and when looking at it with a
text editor, it is just nil characters added to the database after the
actual data, like something went into a loop and added records until the
max size of 4 gb is struck.
I looked at my code and there is nothing that can cause this sort of
behaviour, the same app is working at many sites and this is the only the
second one doing it, both on a Windows 2008 server.

Hope this helps.
Ben

What does record count say? Any indexes? They grow too??

Jim

On 05/26/2012 03:41 AM, Ben Bodenstein wrote:
>
> Hi all,
>
> Thanks for all the replies. Maybe I didn't make myself clear, it is not the
> data that is getting to 4 GB, the data is only a few 100 kb's.
> Something causes the database to grow to 4 GB and when looking at it with a
> text editor, it is just nil characters added to the database after the
> actual data, like something went into a loop and added records until the
> max size of 4 gb is struck.
> I looked at my code and there is nothing that can cause this sort of
> behaviour, the same app is working at many sites and this is the only the
> second one doing it, both on a Windows 2008 server.
>
> Hope this helps.
> Ben

Hi James,

No, the indexes stays as is and the DBF is corrupted, so it is not
accessable to check record count.

Regards,
Ben

On Sat, 26 May 2012 17:07:57 -0400, James Loughner wrote:

> What does record count say? Any indexes? They grow too??
> 
> Jim
> 
> On 05/26/2012 03:41 AM, Ben Bodenstein wrote:
>>
>> Hi all,
>>
>> Thanks for all the replies. Maybe I didn't make myself clear, it is not the
>> data that is getting to 4 GB, the data is only a few 100 kb's.
>> Something causes the database to grow to 4 GB and when looking at it with a
>> text editor, it is just nil characters added to the database after the
>> actual data, like something went into a loop and added records until the
>> max size of 4 gb is struck.
>> I looked at my code and there is nothing that can cause this sort of
>> behaviour, the same app is working at many sites and this is the only the
>> second one doing it, both on a Windows 2008 server.
>>
>> Hope this helps.
>> Ben

Op 26-5-2012 9:41, Ben Bodenstein schreef:
>
> Hi all,
>
> Thanks for all the replies. Maybe I didn't make myself clear, it is not the
> data that is getting to 4 GB, the data is only a few 100 kb's.
> Something causes the database to grow to 4 GB and when looking at it with a
> text editor, it is just nil characters added to the database after the
> actual data, like something went into a loop and added records until the
> max size of 4 gb is struck.
> I looked at my code and there is nothing that can cause this sort of
> behaviour, the same app is working at many sites and this is the only the
> second one doing it, both on a Windows 2008 server.
>
> Hope this helps.
> Ben

Hello Ben,

We have had the same problem for as long as I can remember, although it 
is happening a lot less now, but I guess this is mainly because our 
support-team has applied correct network settings at client-sites. As 
far as we know, the main reason is always wrong Windows settings, like 
SMB, OPLocking, caching, virusscanners etc, the usual. I'm not going 
into that, let's focus on the 4GB.
This always happens a lot more if the table stays open for a long period 
of time and we think it happens when Appending records to the table.
The reason why this happens is that the record-count, for unknown 
reasons, gets corrupted and then it tries to write at 
recordsize*recordcount which is > 4GB, resulting in empty data in 
between (and a lost last record).
But I have no idea when this happens and why, probably just corrupted 
data due to above reasons.

Fortunately this is almost always fixable by doing the reverse, writing 
the correct number of records and packing.
So I can't tell you how to make sure it never happens again, but I can 
tell you how to fix it:

In our software I have written a fix-tool that checks all databases for 
any-kind of corruption, including corrupted headers, corrupted 
memo-files etc. Our DB-structure is available in memory, so we can 
always rebuild an empty table and replace the headers and then fill in 
the variables like recordcount and auto-inrement values.
But if you haven't got your DB-structure, another way to fix it is to 
open the file for write access using FOpen() and then write a large 
number of records, but make sure the number of records is larger then 
the number of records that should be in your table.
I generally use 2.000.000, we don't have tables with that much records.
This makes the file readable by the DB engine again (but with a lot of 
empty records at the end) and after packing the table, the number of 
records will be overwritten with the correct number of records.

Check the DBF specs for where to write: 
http://msdn.microsoft.com/en-us/library/aa975386%28v=vs.71%29.aspx

You can write the number of records manually using a text-editor in 
HEX-mode or in code with FOpen(), FSeek( nH, 4) and then convert the 
number of records (or 2 million) using PadL( NtoC( nRecCount, 16 ), 8, 
"0" ) and FWrite() that to your DBF (least significant byte first, so 
actually the reverse of the NtoC conversion). After packing it you can 
check the count again to see if it worked.

Concluding: apply correct network settings and try not to keep your 
files open for a long period of time. If it keeps happening, you can 
build a fix in your software.
We are having around 800 sites with up to 10 users per site and 
currently we are having this problem around once every 2-3 weeks...

Thank you Sebastiaan for your reply.
Your detailed explanation is exactly what I experienced and it is
definitely happening when the tables have stayed open for a long time, like
a few days.
I will now close the tables every day to prevent this from happening again.

Thanks again for your tips,
Ben


On Tue, 29 May 2012 15:20:37 +0200, Sebastiaan wrote:

> Op 26-5-2012 9:41, Ben Bodenstein schreef:
>>
>> Hi all,
>>
>> Thanks for all the replies. Maybe I didn't make myself clear, it is not the
>> data that is getting to 4 GB, the data is only a few 100 kb's.
>> Something causes the database to grow to 4 GB and when looking at it with a
>> text editor, it is just nil characters added to the database after the
>> actual data, like something went into a loop and added records until the
>> max size of 4 gb is struck.
>> I looked at my code and there is nothing that can cause this sort of
>> behaviour, the same app is working at many sites and this is the only the
>> second one doing it, both on a Windows 2008 server.
>>
>> Hope this helps.
>> Ben
> 
> Hello Ben,
> 
> We have had the same problem for as long as I can remember, although it 
> is happening a lot less now, but I guess this is mainly because our 
> support-team has applied correct network settings at client-sites. As 
> far as we know, the main reason is always wrong Windows settings, like 
> SMB, OPLocking, caching, virusscanners etc, the usual. I'm not going 
> into that, let's focus on the 4GB.
> This always happens a lot more if the table stays open for a long period 
> of time and we think it happens when Appending records to the table.
> The reason why this happens is that the record-count, for unknown 
> reasons, gets corrupted and then it tries to write at 
> recordsize*recordcount which is > 4GB, resulting in empty data in 
> between (and a lost last record).
> But I have no idea when this happens and why, probably just corrupted 
> data due to above reasons.
> 
> Fortunately this is almost always fixable by doing the reverse, writing 
> the correct number of records and packing.
> So I can't tell you how to make sure it never happens again, but I can 
> tell you how to fix it:
> 
> In our software I have written a fix-tool that checks all databases for 
> any-kind of corruption, including corrupted headers, corrupted 
> memo-files etc. Our DB-structure is available in memory, so we can 
> always rebuild an empty table and replace the headers and then fill in 
> the variables like recordcount and auto-inrement values.
> But if you haven't got your DB-structure, another way to fix it is to 
> open the file for write access using FOpen() and then write a large 
> number of records, but make sure the number of records is larger then 
> the number of records that should be in your table.
> I generally use 2.000.000, we don't have tables with that much records.
> This makes the file readable by the DB engine again (but with a lot of 
> empty records at the end) and after packing the table, the number of 
> records will be overwritten with the correct number of records.
> 
> Check the DBF specs for where to write: 
> http://msdn.microsoft.com/en-us/library/aa975386%28v=vs.71%29.aspx
> 
> You can write the number of records manually using a text-editor in 
> HEX-mode or in code with FOpen(), FSeek( nH, 4) and then convert the 
> number of records (or 2 million) using PadL( NtoC( nRecCount, 16 ), 8, 
> "0" ) and FWrite() that to your DBF (least significant byte first, so 
> actually the reverse of the NtoC conversion). After packing it you can 
> check the count again to see if it worked.
> 
> Concluding: apply correct network settings and try not to keep your 
> files open for a long period of time. If it keeps happening, you can 
> build a fix in your software.
> We are having around 800 sites with up to 10 users per site and 
> currently we are having this problem around once every 2-3 weeks...

Any chance that someone is going to hibernate or suspend with open 
DBF's?  I have seen odd things happen coming out of a hibernate though 
never a 4gig file.

Jim


On 06/10/2012 06:30 AM, Ben Bodenstein wrote:
>
> Thank you Sebastiaan for your reply.
> Your detailed explanation is exactly what I experienced and it is
> definitely happening when the tables have stayed open for a long time, like
> a few days.
> I will now close the tables every day to prevent this from happening again.
>
> Thanks again for your tips,
> Ben
>
>
> On Tue, 29 May 2012 15:20:37 +0200, Sebastiaan wrote:
>
>> Op 26-5-2012 9:41, Ben Bodenstein schreef:
>>>
>>> Hi all,
>>>
>>> Thanks for all the replies. Maybe I didn't make myself clear, it is not the
>>> data that is getting to 4 GB, the data is only a few 100 kb's.
>>> Something causes the database to grow to 4 GB and when looking at it with a
>>> text editor, it is just nil characters added to the database after the
>>> actual data, like something went into a loop and added records until the
>>> max size of 4 gb is struck.
>>> I looked at my code and there is nothing that can cause this sort of
>>> behaviour, the same app is working at many sites and this is the only the
>>> second one doing it, both on a Windows 2008 server.
>>>
>>> Hope this helps.
>>> Ben
>>
>> Hello Ben,
>>
>> We have had the same problem for as long as I can remember, although it
>> is happening a lot less now, but I guess this is mainly because our
>> support-team has applied correct network settings at client-sites. As
>> far as we know, the main reason is always wrong Windows settings, like
>> SMB, OPLocking, caching, virusscanners etc, the usual. I'm not going
>> into that, let's focus on the 4GB.
>> This always happens a lot more if the table stays open for a long period
>> of time and we think it happens when Appending records to the table.
>> The reason why this happens is that the record-count, for unknown
>> reasons, gets corrupted and then it tries to write at
>> recordsize*recordcount which is>  4GB, resulting in empty data in
>> between (and a lost last record).
>> But I have no idea when this happens and why, probably just corrupted
>> data due to above reasons.
>>
>> Fortunately this is almost always fixable by doing the reverse, writing
>> the correct number of records and packing.
>> So I can't tell you how to make sure it never happens again, but I can
>> tell you how to fix it:
>>
>> In our software I have written a fix-tool that checks all databases for
>> any-kind of corruption, including corrupted headers, corrupted
>> memo-files etc. Our DB-structure is available in memory, so we can
>> always rebuild an empty table and replace the headers and then fill in
>> the variables like recordcount and auto-inrement values.
>> But if you haven't got your DB-structure, another way to fix it is to
>> open the file for write access using FOpen() and then write a large
>> number of records, but make sure the number of records is larger then
>> the number of records that should be in your table.
>> I generally use 2.000.000, we don't have tables with that much records.
>> This makes the file readable by the DB engine again (but with a lot of
>> empty records at the end) and after packing the table, the number of
>> records will be overwritten with the correct number of records.
>>
>> Check the DBF specs for where to write:
>> http://msdn.microsoft.com/en-us/library/aa975386%28v=vs.71%29.aspx
>>
>> You can write the number of records manually using a text-editor in
>> HEX-mode or in code with FOpen(), FSeek( nH, 4) and then convert the
>> number of records (or 2 million) using PadL( NtoC( nRecCount, 16 ), 8,
>> "0" ) and FWrite() that to your DBF (least significant byte first, so
>> actually the reverse of the NtoC conversion). After packing it you can
>> check the count again to see if it worked.
>>
>> Concluding: apply correct network settings and try not to keep your
>> files open for a long period of time. If it keeps happening, you can
>> build a fix in your software.
>> We are having around 800 sites with up to 10 users per site and
>> currently we are having this problem around once every 2-3 weeks...

Op 11-6-2012 23:48, James Loughner schreef:
>
> Any chance that someone is going to hibernate or suspend with open
> DBF's? I have seen odd things happen coming out of a hibernate though
> never a 4gig file.
>
> Jim
>
>

That has always been someting I suspected, but I have not been able to 
confirm it. It happens while working during the day, not only when 
resuming work, but it is possible that, for example PC1 is working and 
when PC2 resumes from hibernate -> corrupted DBF.
But I'm not sure if this only happens when hibernating or sleeping, I 
think networkcards going into sleep mode will result in the same thing. 
We always disable networkcards going into sleep mode and check again if 
we see DBFs growing to 4GB, in most cases this prevents it from 
happening again..
But I don't know how to prevent it from happening, even if hibernates 
are the problem...

Ben Bodenstein <ben@powerprop.co.za> wrote in message
news:1mts4rhabjxbu.wjmp758railo.dlg@40tude.net...
>Hi all,
>
>Did anyone have the experience where a DBF grows to 4 GB?
>I am running my apps on a Windows Server 2008 with about 20 users where
>this now happened for the 3rd time on two different databases. The
>databases are completely corrupt and I either had to resolve to backups or
>salvage the data in the hard way by editing the database in text format.
>
>Any ideas why this happened please.
>
>Ben

Ben,

I too have had many issues with this.

I would suggest that you add the following code to your "event loop":

if nEvent == xbeP_SystemPowerStatus  && pc is resuming from hibernation

     display a message to the user that he is being shut down
*
  quit      && just quit
endif

Hi,

  Yes, I have this experience...
  It ocurr when you use mapped drive + Terminal Server (Thin Client)....

  Solution:
  (1) Dont use mapped drive, only real drive (C:, D:, ....)
  (2) Use Advantage database (ADS)

  Att
  Osvaldo

"Ben Bodenstein"  escreveu na notícia da 
mensagem:1mts4rhabjxbu.wjmp758railo.dlg@40tude.net...

Hi all,

Did anyone have the experience where a DBF grows to 4 GB?
I am running my apps on a Windows Server 2008 with about 20 users where
this now happened for the 3rd time on two different databases. The
databases are completely corrupt and I either had to resolve to backups or
salvage the data in the hard way by editing the database in text format.

Any ideas why this happened please.

Ben