Hello!

when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
problems with smaler files.

the connection goes through firewalls and over vpn (ipsec).
no communication problems with many other programs like pcanywhere, 
filezilla or copy the zip file with ftp at win xp cmd level.

has someone an idea?

i use xbase++ 1.90.355

Regards,
Roland

Roland Gentner wrote:

> when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
> problems with smaler files.
[...]
> 
> has someone an idea?

What does the FTP server log say?

You could also try to tap into the connection by using Wireshark and see
what goes wrong...

Thomas

Hi
if you have ipsec vpn and your firewall configured or desatived, you can use 
copy file to transfer

"Roland Gentner" <rg@gentner.net> escribió en el mensaje de 
noticias:4672c9ca$1a34d20b$55040@news.alaska-software.com...
> Hello!
>
> when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
> problems with smaler files.
>
> the connection goes through firewalls and over vpn (ipsec).
> no communication problems with many other programs like pcanywhere, 
> filezilla or copy the zip file with ftp at win xp cmd level.
>
> has someone an idea?
>
> i use xbase++ 1.90.355
>
> Regards,
> Roland
>
>

Hello Jorge

you are right in this case i can use copy.
but vpn is only for our program tests.
the module should work with ftp. later - in life mode - our clints havent 
vpn.

Roland

"Jorge L" <jlborlando@way.com.ar> schrieb im Newsbeitrag 
news:70ed3e7$15f3354f$60c79@news.alaska-software.com...
> Hi
> if you have ipsec vpn and your firewall configured or desatived, you can 
> use copy file to transfer
>
> "Roland Gentner" <rg@gentner.net> escribi en el mensaje de 
> noticias:4672c9ca$1a34d20b$55040@news.alaska-software.com...
>> Hello!
>>
>> when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
>> problems with smaler files.
>>
>> the connection goes through firewalls and over vpn (ipsec).
>> no communication problems with many other programs like pcanywhere, 
>> filezilla or copy the zip file with ftp at win xp cmd level.
>>
>> has someone an idea?
>>
>> i use xbase++ 1.90.355
>>
>> Regards,
>> Roland
>>
>>
>

Hi Roland,
i don't know asinet, but i use xbftp of phil ide, i transfer zip files of 
10gb o more without problems

"Roland Gentner" <rg@gentner.net> escribió en el mensaje de 
noticias:78653485$bc5fa83$609d7@news.alaska-software.com...
> Hello Jorge
>
> you are right in this case i can use copy.
> but vpn is only for our program tests.
> the module should work with ftp. later - in life mode - our clints havent 
> vpn.
>
> Roland
>
> "Jorge L" <jlborlando@way.com.ar> schrieb im Newsbeitrag 
> news:70ed3e7$15f3354f$60c79@news.alaska-software.com...
>> Hi
>> if you have ipsec vpn and your firewall configured or desatived, you can 
>> use copy file to transfer
>>
>> "Roland Gentner" <rg@gentner.net> escribió en el mensaje de 
>> noticias:4672c9ca$1a34d20b$55040@news.alaska-software.com...
>>> Hello!
>>>
>>> when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
>>> problems with smaler files.
>>>
>>> the connection goes through firewalls and over vpn (ipsec).
>>> no communication problems with many other programs like pcanywhere, 
>>> filezilla or copy the zip file with ftp at win xp cmd level.
>>>
>>> has someone an idea?
>>>
>>> i use xbase++ 1.90.355
>>>
>>> Regards,
>>> Roland
>>>
>>>
>>
>
>
>

Roland Gentner wrote:

> you are right in this case i can use copy.
> but vpn is only for our program tests.
> the module should work with ftp. later - in life mode - our clints havent 
> vpn.

Are you shure that the problem is related to/caused by the VPN?

Thomas

Hi Thomas
i have 5 vpn's with 5 clients

i was configured the vpn with ruters dlink 804, 808, 824, and ipsec

the ftp server is made with windows 2003 server (anonymous user), and linux 
ubuntu 9.01 (in my case with user and password) and i have a ftp server in 
"towebs"

i haven´t problem if
             i can make a "xcopy" command or
             i can download a file with ftp.exe command






"Thomas Braun" <spam@software-braun.de> escribió en el mensaje de 
noticias:1rz8m391tt2j1$.ydeqs0nt5zjk$.dlg@40tude.net...
> Roland Gentner wrote:
>
>> you are right in this case i can use copy.
>> but vpn is only for our program tests.
>> the module should work with ftp. later - in life mode - our clints havent
>> vpn.
>
> Are you shure that the problem is related to/caused by the VPN?
>
> Thomas
>

Hi Thomas again

Is the time of life of the vpn superior to the necessary time to transfer 
the file ?



"Thomas Braun" <spam@software-braun.de> escribió en el mensaje de 
noticias:1rz8m391tt2j1$.ydeqs0nt5zjk$.dlg@40tude.net...
> Roland Gentner wrote:
>
>> you are right in this case i can use copy.
>> but vpn is only for our program tests.
>> the module should work with ftp. later - in life mode - our clints havent
>> vpn.
>
> Are you shure that the problem is related to/caused by the VPN?
>
> Thomas
>

Jorge L wrote:

> Hi Thomas again
> 
> Is the time of life of the vpn superior to the necessary time to transfer 
> the file ?

Of course not - this is only an inactivity timeout... as long as there is
data transfered, the connection should not drop.

But without more information from Roland, we can only guess 

Thomas

Hello Thomas,

vpn again (exactly we talk about IPSec):
basicaly ipsec itself hasnt any effect on data transfer timeout or data 
transfer inactivity.
if you configure key exchange initiated after a defined ammount of data 
transfer (i.e. 1MB) a key exchange will be started.
maybe there are additional parameters on the machine you have configured 
vpn.
if the defined encryption is to strong there can be communication problems 
at key exchange or when the ipsec "overhead" is to big the realy ammount of 
transfered data is too low (syncronisation problems between the 
applications).
we run and configured many vpns since years (self and clients)
...stop vpn explanation - we have a ftp problem 
so i didnt think the ftp abort is a vpn problem.

Roland

"Thomas Braun" <spam@software-braun.de> schrieb im Newsbeitrag 
news:hwvu5gjim227$.10dolvsmm0eid$.dlg@40tude.net...
> Jorge L wrote:
>
>> Hi Thomas again
>>
>> Is the time of life of the vpn superior to the necessary time to transfer
>> the file ?
>
> Of course not - this is only an inactivity timeout... as long as there is
> data transfered, the connection should not drop.
>
> But without more information from Roland, we can only guess 
>
> Thomas

Hello Thomas,

no i didnt think it is a vpn problem.
i only list up my way of communicate with the web server.

meanwhile we have a new firewall and new vpn but have the same problem.

Roland

"Thomas Braun" <spam@software-braun.de> schrieb im Newsbeitrag 
news:1rz8m391tt2j1$.ydeqs0nt5zjk$.dlg@40tude.net...
> Roland Gentner wrote:
>
>> you are right in this case i can use copy.
>> but vpn is only for our program tests.
>> the module should work with ftp. later - in life mode - our clints havent
>> vpn.
>
> Are you shure that the problem is related to/caused by the VPN?
>
> Thomas

Roland Gentner wrote:

> no i didnt think it is a vpn problem.

So you can reproduce this with a non-VPN connection as well?

Thomas

Roland,

>when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
>problems with smaler files.

>the connection goes through firewalls and over vpn (ipsec).
>no communication problems with many other programs like pcanywhere, 
>filezilla or copy the zip file with ftp at win xp cmd level.

FTP connections consist actually of two separate connections. The FTP 
command connection, which is usually on port 21 or 990 (for FTPS) and 
the data transfer connection which is on a different (higher) port. 
Each individual file transfer might actually be on a different port 
within a configured port range. If you use passive mode, the FTP Server 
will tell your FTP Client which port to use when starting a transfer.

The command connection on Port 21 or 990 -- or whatever port you might
use -- might time-out during lengthy data transfers. This time-out might
be a setting within a hub/switch/router/firewall or VPN. This problem is
not an issue with FileZilla or the MS command line ftp.exe program, as 
FileZilla will automatically re-connect a lost connection, while ftp.exe
doesn't give a <#$@!> if there is a command connection or not.

You should examine your Server's and (your application's) FTP logs, to 
see at what time the abort/disconnect happens. It should always be after
a fixed amount of time has passed since the transfer started, like after 
10 minutes or after an hour, etc. Then you could examine all the places 
a time-out is specified, including appliances and software firewalls and
the VPN Client, as well as your FTP Server. If you use a non-standard FTP 
command port (other than 21) there might be no way to change the time-out 
settings, though. But most appliances don't time-out port 21 (for at least 
an hour or so), so using port 21 might be the simplest solution, if you 
don't already use it.

Hope that helps.

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas_Gehrs-Pahl@CrimeCog.com
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Andreas Gehrs-Pahl wrote:

> settings, though. But most appliances don't time-out port 21 (for at least 
> an hour or so), so using port 21 might be the simplest solution, if you 
> don't already use it.

I'm not sure what ASINET is using, but maybe using passive FTP instead of
active FTP might be an option too.

Or Roland could do it like me - I'm using WinSCP as the FTP client, which
is not only free, but also scriptable. I create a FTP-Script and call
WinSCP with the script name.

Thomas

Thomas,

>I'm not sure what ASINET is using, but maybe using passive FTP instead of
>active FTP might be an option too.

I don't know what ASINet uses either -- and you can't change it anyway -- 
but it makes no difference as far as the command connection timing out is
concerned. ASINet (as well as Xb2Net and FCE32 for that matter) all behave 
the same way in that respect. None of theme have an asynchronous keep-
alive method for the command connection while a transfer is in progress. 

I personally don't use ASINet (for FTP) as it has too many limitations, 
which make it basically useless in the real world. I use Marshall Soft's 
FCE32 for basic FTP and Boris' Xb2Net for secure FTPS. I would probably 
switch completely to Xb2Net, if it would (fully) support append mode. I 
(virtually) always use passive mode, to get around some firewall issues,
but I have seen those time-outs, too, especially when using port 990.

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas_Gehrs-Pahl@CrimeCog.com
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Andreas Gehrs-Pahl wrote in
news:phsytdpygx5h.1gpfgjjek52hd.dlg@40tude.net: 

> I use Marshall Soft's FCE32 for basic FTP and Boris' Xb2Net for secure
> FTPS. I would probably switch completely to Xb2Net, if it would
> (fully) support append mode. 

FYI... a new <lAppend> param was added to xbFTPClient:GetFile and 
xbFTPClient:PutFile in release ver 3.2.13.

Best regards,
Boris Borzic

http://xb2.net
http://sqlexpress.net
industrial strength Xbase++ development tools

Boris,

>>I use Marshall Soft's FCE32 for basic FTP and Boris' Xb2Net for secure
>>FTPS. I would probably switch completely to Xb2Net, if it would
>>(fully) support append mode. 

>FYI... a new <lAppend> param was added to xbFTPClient:GetFile and 
>xbFTPClient:PutFile in release ver 3.2.13.

I know, but this is not enough to fully support append mode. What is 
needed in addition to the <lAppend> parameter is an <nOffset> parameter
that starts the sending from a different, advanced, file position, 
rather than from the beginning of the file. 

Without that, you can append to an existing file, but you can't resume
an interrupted transfer at a given position!

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas_Gehrs-Pahl@CrimeCog.com
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Andreas Gehrs-Pahl wrote in news:185qfyygjcvzl.amj93xxr6yvu.dlg@40tude.net:

>>FYI... a new <lAppend> param was added to xbFTPClient:GetFile and 
>>xbFTPClient:PutFile in release ver 3.2.13.
> 
> I know, but this is not enough to fully support append mode. What is 
> needed in addition to the <lAppend> parameter is an <nOffset> parameter
> that starts the sending from a different, advanced, file position, 
> rather than from the beginning of the file. 
> 
> Without that, you can append to an existing file, but you can't resume
> an interrupted transfer at a given position!

Try this:

1. Upload to FTP server:
:PutFile(xLocalFile, [cRemoteFile], [cDataType], [lAppend])

Pass xLocalFile as a numeric file handle. Use FSEEK to position file offset 
to position that you want to start uploading.

You can determine size of the partially uploaded file on server by using 
:Directory() or :SendCommand("SIZE " + cRemoteFile)

2. Download from FTP server:
:SendCommand("REST " + ltrim(str(nOffset)))
:GetFile(cRemoteFile, [xLocalFile], [cDataType], [lAppend])

The nOffset will be the current size of the local file.

Best regards,
Boris Borzic

http://xb2.net
http://sqlexpress.net
industrial strength Xbase++ development tools

Boris,

>Try this:

>1. Upload to FTP server:
>:PutFile(xLocalFile, [cRemoteFile], [cDataType], [lAppend])

>Pass xLocalFile as a numeric file handle. Use FSEEK to position file offset 
>to position that you want to start uploading.

>You can determine size of the partially uploaded file on server by using 
>:Directory() or :SendCommand("SIZE " + cRemoteFile)

>2. Download from FTP server:
>:SendCommand("REST " + ltrim(str(nOffset)))
>:GetFile(cRemoteFile, [xLocalFile], [cDataType], [lAppend])

>The nOffset will be the current size of the local file.

This should work. Do you have any plans to directly integrate this with
Xb2Net, to give us something simple like this:

:PutFile(xLocalFile, [cRemoteFile], [cDataType], [lAppend], [nOffset])
:GetFile(cRemoteFile, [xLocalFile], [cDataType], [lAppend], [nOffset])

It would make (my) code much cleaner and simpler. For now, I will see 
if I can sub-class this to get the same effect, and then I will do some 
testing to see if there are any unforeseen side effects.

Thanks,

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas_Gehrs-Pahl@CrimeCog.com
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Andreas Gehrs-Pahl wrote in news:m8dfvzg2zp6h$.w8uyitall487$.dlg@
40tude.net:

> This should work. Do you have any plans to directly integrate this with
> Xb2Net, to give us something simple like this:
>
>:PutFile(xLocalFile, [cRemoteFile], [cDataType], [lAppend], [nOffset])
>:GetFile(cRemoteFile, [xLocalFile], [cDataType], [lAppend], [nOffset])
> 
> It would make (my) code much cleaner and simpler. For now, I will see 
> if I can sub-class this to get the same effect, and then I will do some 
> testing to see if there are any unforeseen side effects.

Happy to add this. I'm thinking that the nOffset param is not needed. The 
methods should figure out the required offset. Maybe the new param should 
be lResume - or maybe we can assume that a transmission is resumed if 
lAppend is .T. - therefore no new param?

Show me the design that you end up with. 

Best regards,
Boris Borzic

http://xb2.net
http://sqlexpress.net
industrial strength Xbase++ development tools

Boris,

>Happy to add this. I'm thinking that the nOffset param is not needed. The 
>methods should figure out the required offset. Maybe the new param should 
>be lResume - or maybe we can assume that a transmission is resumed if 
>lAppend is .T. - therefore no new param?

I think two separate options should be available, as I use this both ways 
in my applications. 

I use append without a (local) offset to append a new log entry to the end 
of an existing log file, and I also use append with an offset to continue 
an interrupted transfer (both directions). I either automatically resume 
or replace -- or I might ask the user if the file should be replaced or if
the transfer should be resumed -- if I find an exiting file on the server 
(or locally) that is smaller than the file that I want to transfer. This
depends on the situation and the user's access rights -- because a replace
might not be allowed, so the choice might be just between resume, rename, 
or cancel.

>Show me the design that you end up with.

I will do that. Right now, I'm preparing for the upcoming Michigan Chief's 
of Police conference at the end of June, but I will get back to this topic
at the beginning of July. I will email you my routines as soon as I have 
them working to my satisfaction, probably sometime in early to mid July.

-- Andreas

---                                                                      ---
  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
  415 Gute Street                     or: Andreas@DDPSoftware.com
  Owosso, MI 48867-4410               or: Andreas_Gehrs-Pahl@CrimeCog.com
  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
---                                                                      ---

Andreas Gehrs-Pahl wrote:

> I personally don't use ASINet (for FTP) as it has too many limitations, 
> which make it basically useless in the real world. 

That's why I use WinSCP via scripting. It is not also a perfect solution
when "nice" error handling is needed, but I use SSH file tansfer a lot so
for my needs it works flawless - plus its free 

Thomas

Hi Thomas

I occasionally get transmit errors with Xb2Net FTP, even if I try 10 
times in an attempt to make a transfer more error-proof. In your 
opinion, would WinScp be more likely to get a file through via FTP than 
Xb2Net?

For example are there situations where you have seen Xb2net fail and 
WinScp succeed?

Cheers

Phil Jackson

On 6/11/2010 12:10 AM, Thomas Braun wrote:
> Andreas Gehrs-Pahl wrote:
>
>> I personally don't use ASINet (for FTP) as it has too many limitations,
>> which make it basically useless in the real world.
>
> That's why I use WinSCP via scripting. It is not also a perfect solution
> when "nice" error handling is needed, but I use SSH file tansfer a lot so
> for my needs it works flawless - plus its free 
>
> Thomas

PhilJackson wrote:

> I occasionally get transmit errors with Xb2Net FTP, even if I try 10 
> times in an attempt to make a transfer more error-proof. In your 
> opinion, would WinScp be more likely to get a file through via FTP than 
> Xb2Net?

I don't think so - but I'm not using Xb2Net anyway, so I might not be the
best source to be asked 

Just to give you an idea... my SSH file transfer jobs scripts are looking
like this:

option batch on
option confirm off
option synchdelete off
open <put the name of your target profile here>

put <source file> <targetfile>
... multiple lines

exit

Then I use runshell to call the script:

xRet :=  RunShell( "/console /log=D:\wwwroot\winscp_session.log /script=D:\wwwroot\upload.scp",  "C:\Programme\winscp3\WinSCP.exe" )
IF Valtype(xRet) = "N"
   xRet := LTRIM(STR(xRet))
ENDIF

IF xRet != "0"
   ?
   ? "Transmission error... see D:\wwwroot\hp2\winscp_session.log for more details."
   ? "Press any key to continue."

   INKEY(0)
ENDIF


Thomas

Hello Andreas,

the transfer breaks not at the same ammount of transfered data.
i will stop time from start to abortion.

Roland

<Andreas Gehrs-Pahl> schrieb im Newsbeitrag 
news:1vtto6r6d3nf6$.a8vvul5c6g5j$.dlg@40tude.net...
> Roland,
>
>>when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no
>>problems with smaler files.
>
>>the connection goes through firewalls and over vpn (ipsec).
>>no communication problems with many other programs like pcanywhere,
>>filezilla or copy the zip file with ftp at win xp cmd level.
>
> FTP connections consist actually of two separate connections. The FTP
> command connection, which is usually on port 21 or 990 (for FTPS) and
> the data transfer connection which is on a different (higher) port.
> Each individual file transfer might actually be on a different port
> within a configured port range. If you use passive mode, the FTP Server
> will tell your FTP Client which port to use when starting a transfer.
>
> The command connection on Port 21 or 990 -- or whatever port you might
> use -- might time-out during lengthy data transfers. This time-out might
> be a setting within a hub/switch/router/firewall or VPN. This problem is
> not an issue with FileZilla or the MS command line ftp.exe program, as
> FileZilla will automatically re-connect a lost connection, while ftp.exe
> doesn't give a <#$@!> if there is a command connection or not.
>
> You should examine your Server's and (your application's) FTP logs, to
> see at what time the abort/disconnect happens. It should always be after
> a fixed amount of time has passed since the transfer started, like after
> 10 minutes or after an hour, etc. Then you could examine all the places
> a time-out is specified, including appliances and software firewalls and
> the VPN Client, as well as your FTP Server. If you use a non-standard FTP
> command port (other than 21) there might be no way to change the time-out
> settings, though. But most appliances don't time-out port 21 (for at least
> an hour or so), so using port 21 might be the simplest solution, if you
> don't already use it.
>
> Hope that helps.
>
> -- Andreas
>
> ---                                                                      ---
>  Andreas Gehrs-Pahl              E-Mail: GPahl@CharterMI.net
>  415 Gute Street                     or: Andreas@DDPSoftware.com
>  Owosso, MI 48867-4410               or: Andreas_Gehrs-Pahl@CrimeCog.com
>  Tel: (989) 723-9927           Web Site: http://www.Aerospace-History.net
> ---                                                                      ---

hi Roland

Probably could implement a solution until you discover the solution to the 
problem
 1 - it is not to depend on the ftp between the vpn
 2 - in your :Init() class to do a "ping" to the "host" and if it exists, 
your :send() or Recived() method, to use a "copy to " instead asinet




"Roland Gentner" <rg@gentner.net> escribió en el mensaje de 
noticias:4672c9ca$1a34d20b$55040@news.alaska-software.com...
> Hello!
>
> when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
> problems with smaler files.
>
> the connection goes through firewalls and over vpn (ipsec).
> no communication problems with many other programs like pcanywhere, 
> filezilla or copy the zip file with ftp at win xp cmd level.
>
> has someone an idea?
>
> i use xbase++ 1.90.355
>
> Regards,
> Roland
>
>

I had lots of problems until I changed to PASV (Passive) mode as the
default 

"Roland Gentner" <rg@gentner.net> wrote:

>Hello!
>
>when i transfer zip-files i.e 5mb the ftp transfer abort at 3 to 4mb, no 
>problems with smaler files.
>
>the connection goes through firewalls and over vpn (ipsec).
>no communication problems with many other programs like pcanywhere, 
>filezilla or copy the zip file with ftp at win xp cmd level.
>
>has someone an idea?
>
>i use xbase++ 1.90.355
>
>Regards,
>Roland 
>