I have a file to be retrieved from Web, what I can do very easily with 
LoadFromURL.

The problem is that I have to pull an elephant across a swamp !

That is, the file is huge and, the band is slow.

So, I want to prevent it from being pulled every time. Just want to download 
it when it has been updated at the site.

When regarded to local files, I do the same thing very often, like this (sorry 
about formatting) :

IF FILETIME("my_local_version_of_the_file_on_a_ramdrive") 
               is not equal to FILETIME("original_file_on_a_NOVELL_netware") THEN
FILECOPY("original_file_on_a_NOVELL_netware","my_local_version_of_the_file_on_a_ramdrive")
END IF

And I go on using my local file. I notice that this offers a huge improvement 
on perfomance, of course. Using a local file is way, way faster than accessing 
it remotelly.

I'd like to do the same checking (FILETIME) with LoadFromURL.

We cant do that with ASINET FTP Client either because it cant retrieve 
diretory information.

Some ideas ?

thanXs in advance for your suPPOrt

On Fri, 14 Jan 2005 10:15:44 -0200, Jose Adriano Baltieri wrote:

> I have a file to be retrieved from Web, what I can do very easily with 
> LoadFromURL.
> 
> The problem is that I have to pull an elephant across a swamp !
> 
> That is, the file is huge and, the band is slow.
> 
> So, I want to prevent it from being pulled every time. Just want to download 
> it when it has been updated at the site.
> 
> When regarded to local files, I do the same thing very often, like this (sorry 
> about formatting) :
> 
> IF FILETIME("my_local_version_of_the_file_on_a_ramdrive") 
>                is not equal to FILETIME("original_file_on_a_NOVELL_netware") THEN
> FILECOPY("original_file_on_a_NOVELL_netware","my_local_version_of_the_file_on_a_ramdrive")
> END IF
> 
> And I go on using my local file. I notice that this offers a huge improvement 
> on perfomance, of course. Using a local file is way, way faster than accessing 
> it remotelly.
> 
> I'd like to do the same checking (FILETIME) with LoadFromURL.
> 
> We cant do that with ASINET FTP Client either because it cant retrieve 
> diretory information.
> 
> Some ideas ?
> 
> thanXs in advance for your suPPOrt

Jose
Have also a versy small file with informations for the huge file in it and
use first the loadfromurl for the small file. These way you can have also
other (server side) notes about the download.

Regards
Mike Evans

Adriano,

> I have a file to be retrieved from Web, what I can do very easily with 
> LoadFromURL.
> 
> The problem is that I have to pull an elephant across a swamp !

If you can FTP, then try my FTP library which should give you date/time
info (although not guaranteed - not all FTP servers supply this info).

However, HTTP/1.1 provides the mechanism you want.  Section 14.25 of
RFC2616 displays this information on an HTTP header (headers are sent along
with the URL to the server). Using my LoadFromURL, you can send this header
with the request, else you easily code it yourself if you have a clue about
HTTP.


14.25 If-Modified-Since

   The If-Modified-Since request-header field is used with a method to
   make it conditional: if the requested variant has not been modified
   since the time specified in this field, an entity will not be
   returned from the server; instead, a 304 (not modified) response will
   be returned without any message-body.

       If-Modified-Since = "If-Modified-Since" ":" HTTP-date

   An example of the field is:

       If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT

   A GET method with an If-Modified-Since header and no Range header
   requests that the identified entity be transferred only if it has
   been modified since the date given by the If-Modified-Since header.
   The algorithm for determining this includes the following cases:

      a) If the request would normally result in anything other than a
         200 (OK) status, or if the passed If-Modified-Since date is
         invalid, the response is exactly the same as for a normal GET.
         A date which is later than the server's current time is
         invalid.

      b) If the variant has been modified since the If-Modified-Since
         date, the response is exactly the same as for a normal GET.

      c) If the variant has not been modified since a valid If-
         Modified-Since date, the server SHOULD return a 304 (Not
         Modified) response.

   The purpose of this feature is to allow efficient updates of cached
   information with a minimum amount of transaction overhead.

      Note: The Range request-header field modifies the meaning of If-
      Modified-Since; see section 14.35 for full details.

      Note: If-Modified-Since times are interpreted by the server, whose
      clock might not be synchronized with the client.

      Note: When handling an If-Modified-Since header field, some
      servers will use an exact date comparison function, rather than a
      less-than function, for deciding whether to send a 304 (Not
      Modified) response. To get best results when sending an If-
      Modified-Since header field for cache validation, clients are
      advised to use the exact date string received in a previous Last-
      Modified header field whenever possible.

      Note: If a client uses an arbitrary date in the If-Modified-Since
      header instead of a date taken from the Last-Modified header for
      the same request, the client should be aware of the fact that this
      date is interpreted in the server's understanding of time. The
      client should consider unsynchronized clocks and rounding problems
      due to the different encodings of time between the client and
      server. This includes the possibility of race conditions if the
      document has changed between the time it was first requested and
      the If-Modified-Since date of a subsequent request, and the
      possibility of clock-skew-related problems if the If-Modified-
      Since date is derived from the client's clock without correction
      to the server's clock. Corrections for different time bases
      between client and server are at best approximate due to network
      latency.

   The result of a request having both an If-Modified-Since header field
   and either an If-Match or an If-Unmodified-Since header fields is
   undefined by this specification.


Regards,

Phil Ide

***************************************
* Xbase++ FAQ, Libraries and Sources: *
* goto: http://www.idep.org.uk/xbase  *
***************************************

FUBB -- Fouled Up Beyond Belief

Phil Ide wrote:
> Adriano,
> 
> 
>>I have a file to be retrieved from Web, what I can do very easily with 
>>LoadFromURL.
>>
>>The problem is that I have to pull an elephant across a swamp !
> 
> 
> If you can FTP, then try my FTP library which should give you date/time
> info (although not guaranteed - not all FTP servers supply this info).
> 
> However, HTTP/1.1 provides the mechanism you want.  Section 14.25 of
> RFC2616 displays this information on an HTTP header (headers are sent along
> with the URL to the server). Using my LoadFromURL, you can send this header
> with the request, else you easily code it yourself if you have a clue about
> HTTP.

Yes, I was here reading my HTTP pocket guide and I also found it.

Have downloaded your LoadFromURL too and was about to start testing. But then, 
I had to go for lunch...

I have to make a small test program first and when this one is OK, put it into 
the real system, which is way more complex.

Have some other suggestions 4 you, Phil :

- You may want to consider another name for your LoadfromURL. I'm copying it 
as PILoadFromURL (Phil Ide). Because I will use it on projects where I already 
use ASINET.

- Have you talked to Alaska about embedding XBPCRE into XBase++ itself ? 
Because it demands some sort of Regex and, AFAIK, PCRE is free to be distributed.

Thanks !

> 
> 14.25 If-Modified-Since
> 
>    The If-Modified-Since request-header field is used with a method to
>    make it conditional: if the requested variant has not been modified
>    since the time specified in this field, an entity will not be
>    returned from the server; instead, a 304 (not modified) response will
>    be returned without any message-body.
> 
>        If-Modified-Since = "If-Modified-Since" ":" HTTP-date
> 
>    An example of the field is:
> 
>        If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT
> 
>    A GET method with an If-Modified-Since header and no Range header
>    requests that the identified entity be transferred only if it has
>    been modified since the date given by the If-Modified-Since header.
>    The algorithm for determining this includes the following cases:
> 
>       a) If the request would normally result in anything other than a
>          200 (OK) status, or if the passed If-Modified-Since date is
>          invalid, the response is exactly the same as for a normal GET.
>          A date which is later than the server's current time is
>          invalid.
> 
>       b) If the variant has been modified since the If-Modified-Since
>          date, the response is exactly the same as for a normal GET.
> 
>       c) If the variant has not been modified since a valid If-
>          Modified-Since date, the server SHOULD return a 304 (Not
>          Modified) response.
> 
>    The purpose of this feature is to allow efficient updates of cached
>    information with a minimum amount of transaction overhead.
> 
>       Note: The Range request-header field modifies the meaning of If-
>       Modified-Since; see section 14.35 for full details.
> 
>       Note: If-Modified-Since times are interpreted by the server, whose
>       clock might not be synchronized with the client.
> 
>       Note: When handling an If-Modified-Since header field, some
>       servers will use an exact date comparison function, rather than a
>       less-than function, for deciding whether to send a 304 (Not
>       Modified) response. To get best results when sending an If-
>       Modified-Since header field for cache validation, clients are
>       advised to use the exact date string received in a previous Last-
>       Modified header field whenever possible.
> 
>       Note: If a client uses an arbitrary date in the If-Modified-Since
>       header instead of a date taken from the Last-Modified header for
>       the same request, the client should be aware of the fact that this
>       date is interpreted in the server's understanding of time. The
>       client should consider unsynchronized clocks and rounding problems
>       due to the different encodings of time between the client and
>       server. This includes the possibility of race conditions if the
>       document has changed between the time it was first requested and
>       the If-Modified-Since date of a subsequent request, and the
>       possibility of clock-skew-related problems if the If-Modified-
>       Since date is derived from the client's clock without correction
>       to the server's clock. Corrections for different time bases
>       between client and server are at best approximate due to network
>       latency.
> 
>    The result of a request having both an If-Modified-Since header field
>    and either an If-Match or an If-Unmodified-Since header fields is
>    undefined by this specification.
> 
> 
> Regards,

Adriano,

> Have downloaded your LoadFromURL too and was about to start testing. But then, 
> I had to go for lunch...
> 
> - You may want to consider another name for your LoadfromURL. I'm copying it 
> as PILoadFromURL (Phil Ide). Because I will use it on projects where I already 
> use ASINET.

Well no, because then it wouldn't be a plug-in replacement for the ASINET
function.  If you add my routine to your application, then it will be used
in preference to the ASINET function anyway.  If you want to use both
versions (I don't see a good reason for that) then you can explicitely cal
the ASINET version using DllCall().

> - Have you talked to Alaska about embedding XBPCRE into XBase++ itself ? 
> Because it demands some sort of Regex and, AFAIK, PCRE is free to be distributed.

Well, I mentioned it some years ago, but yes it would be nice if regex
syntax was supported at the language level.  However, that will require
huge changes to both the compiler and the preprocessor, unless Alaska do
what I have done and include regex via a class.  If they do that, then you
are no better off than you are now - in fact worse, because right now you
have access to the XbPCRE source (so you can change it if you need to), and
if a newer version of the PCRE library becomes available you can just swap
it in.

Regards,

Phil Ide

***************************************
* Xbase++ FAQ, Libraries and Sources: *
* goto: http://www.idep.org.uk/xbase  *
***************************************

If it's tourist season, why can't we shoot 'em??