Alaska Software Inc. - Automated interaction with a web site: possible?
Username: Password:
AuthorTopic: Automated interaction with a web site: possible?
darkeagleAutomated interaction with a web site: possible?
on Mon, 13 Feb 2006 09:17:55 +0100
Hi All,
I have this problem.
I need, from an xbase application, to connect to an url ex www.site.com get
the page containing a java applet, do click on a button in the java applet,
wait for a new response.
The response will contain an HTML form that I need to automatically complete
ad submit. After the submit I need to reprocess the response extracting some
information from HTML tables.
There is anything (library, source etc) that can help me?
Thank you.
Phil Ide
Re: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 09:59:59 +0000
darkeagle,

> I need, from an xbase application, to connect to an url ex www.site.com get
> the page containing a java applet, do click on a button in the java applet,
> wait for a new response.
> The response will contain an HTML form that I need to automatically complete
> ad submit. After the submit I need to reprocess the response extracting some
> information from HTML tables.
> There is anything (library, source etc) that can help me?

Well, your problem really is running the Java.  You could use the IE ActiveX
to fetch and run the page, but then you wil have problems programmatically
pressing the button.

Your better bet would be to load an HTTP packet sniffer and manually
download the page and click the button and see what data it sends and where.

You can then use LoadFromUrl() to download the page and HRF to extract the
data from the table.  Alternatively you could use HRF2 which can do both
actions for you.

Regards,

Phil Ide

***************************************
* Xbase++ FAQ, Libraries and Sources: *
* goto: http://www.idep.org.uk/xbase  *
***************************************

I will be selective in the hiring of assassins. Anyone who attempts to
strike down the hero the first instant his back is turned will not even be
considered for the job.
          [Things I'd do as an Evil Overlord]
darkeagleRe: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 11:35:36 +0100
Hi,
Thank you.
I dont know asinet. LoadFromUrl, HRF, HRF2 are class of asinet?
I have a basic subscription and I can upgrade to prof version to get asinet 
if I think that asinet solve my problem.
I'm evaluating xb2.net. Know this library? What you think about?
Thank you.

"Phil Ide" <phil@idep.org.uk> ha scritto nel messaggio 
news:yx640ai43576.dlg@idep.org.uk...
> darkeagle,
>
>> I need, from an xbase application, to connect to an url ex www.site.com 
>> get
>> the page containing a java applet, do click on a button in the java 
>> applet,
>> wait for a new response.
>> The response will contain an HTML form that I need to automatically 
>> complete
>> ad submit. After the submit I need to reprocess the response extracting 
>> some
>> information from HTML tables.
>> There is anything (library, source etc) that can help me?
>
> Well, your problem really is running the Java.  You could use the IE 
> ActiveX
> to fetch and run the page, but then you wil have problems programmatically
> pressing the button.
>
> Your better bet would be to load an HTTP packet sniffer and manually
> download the page and click the button and see what data it sends and 
> where.
>
> You can then use LoadFromUrl() to download the page and HRF to extract the
> data from the table.  Alternatively you could use HRF2 which can do both
> actions for you.
>
> Regards,
> -- 
> Phil Ide
>
> ***************************************
> * Xbase++ FAQ, Libraries and Sources: *
> * goto: http://www.idep.org.uk/xbase  *
> ***************************************
>
> I will be selective in the hiring of assassins. Anyone who attempts to
> strike down the hero the first instant his back is turned will not even be
> considered for the job.
>          [Things I'd do as an Evil Overlord]
Phil Ide
Re: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 11:58:02 +0000
darkeagle,

> I dont know asinet. LoadFromUrl, HRF, HRF2 are class of asinet?

Well, ASINET contains a LoadFromUrl() function, and the professinal
subscription has HRF (HTML Reflection Framework) which can decompose an HTML
page and extract data.

HRF2 is a 3PP product from Polar Bear International.  HRF2 is like HRF, but
is far more powerful, and it has a LoadFromUrl() function too.

HRF2 retails at 150 euros.  there is  more comprehensive pricelist available
in the generic NG under the tpoic [How many people interested in
"pre-buying" XbHCL w/ training at DevCon?]

Regards,

Phil Ide

***************************************
* Xbase++ FAQ, Libraries and Sources: *
* goto: http://www.idep.org.uk/xbase  *
***************************************

Employees will have conjugal visit trailers which they may use provided they
call in a replacement and sign out on the timesheet. Given this, anyone
caught making out in a closet while leaving their station unmonitored will
be shot.
          [Things I'd do as an Evil Overlord]
darkeagleRe: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 13:05:28 +0100
Hi,
Thank you for patience!!!
Please have a direct link where I can download a demo version of HRF2 or 
where I can buy it?
This library need asinet?
Thank you very much!
D
"Phil Ide" <phil@idep.org.uk> ha scritto nel messaggio 
news:1gr7b4j3167om.dlg@idep.org.uk...
> darkeagle,
>
>> I dont know asinet. LoadFromUrl, HRF, HRF2 are class of asinet?
>
> Well, ASINET contains a LoadFromUrl() function, and the professinal
> subscription has HRF (HTML Reflection Framework) which can decompose an 
> HTML
> page and extract data.
>
> HRF2 is a 3PP product from Polar Bear International.  HRF2 is like HRF, 
> but
> is far more powerful, and it has a LoadFromUrl() function too.
>
> HRF2 retails at 150 euros.  there is  more comprehensive pricelist 
> available
> in the generic NG under the tpoic [How many people interested in
> "pre-buying" XbHCL w/ training at DevCon?]
>
> Regards,
> -- 
> Phil Ide
>
> ***************************************
> * Xbase++ FAQ, Libraries and Sources: *
> * goto: http://www.idep.org.uk/xbase  *
> ***************************************
>
> Employees will have conjugal visit trailers which they may use provided 
> they
> call in a replacement and sign out on the timesheet. Given this, anyone
> caught making out in a closet while leaving their station unmonitored will
> be shot.
>          [Things I'd do as an Evil Overlord]
Phil Ide
Re: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 12:35:28 +0000
darkeagle,

> Thank you for patience!!!
> Please have a direct link where I can download a demo version of HRF2 or 
> where I can buy it?
> This library need asinet?
> Thank you very much!

No, it doesn't need ASINET. Send me a private email and we can talk turkey.

Regards,

Phil Ide

***************************************
* Xbase++ FAQ, Libraries and Sources: *
* goto: http://www.idep.org.uk/xbase  *
***************************************

I will never employ any device with a digital countdown. If I find that such
a device is absolutely unavoidable, I will set it to activate when the
counter reaches 117 and the hero is just putting his plan into operation.
          [Things I'd do as an Evil Overlord]
darkeagleRe: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 14:33:08 +0100
I've sent a private mail.
Please let me know If you do not received.
Thank you
"Phil Ide" <phil@idep.org.uk> ha scritto nel messaggio 
news:17yv5f8zbtxnv$.dlg@idep.org.uk...
> darkeagle,
>
>> Thank you for patience!!!
>> Please have a direct link where I can download a demo version of HRF2 or
>> where I can buy it?
>> This library need asinet?
>> Thank you very much!
>
> No, it doesn't need ASINET. Send me a private email and we can talk 
> turkey.
>
> Regards,
> -- 
> Phil Ide
>
> ***************************************
> * Xbase++ FAQ, Libraries and Sources: *
> * goto: http://www.idep.org.uk/xbase  *
> ***************************************
>
> I will never employ any device with a digital countdown. If I find that 
> such
> a device is absolutely unavoidable, I will set it to activate when the
> counter reaches 117 and the hero is just putting his plan into operation.
>          [Things I'd do as an Evil Overlord]
Phil Ide
Re: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 15:48:22 +0000
darkeagle,

> I've sent a private mail.
> Please let me know If you do not received.

I have received - just knocking up a demo that will help you.

Regards,

Phil Ide

***************************************
* Xbase++ FAQ, Libraries and Sources: *
* goto: http://www.idep.org.uk/xbase  *
***************************************

I will not hold lavish banquets in the middle of a famine. The good PR among
the guests doesn't make up for the bad PR among the masses.
          [Things I'd do as an Evil Overlord]
darkeagleRe: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 17:26:05 +0100
Thank you
As a sample, my xbase application can do a thing like this (all in batch 
mode):
1) connect to www.ebay.it
2) navigate to the login page of ebay,
3) insert username and password and submitting form
4) Navigate to "myEbay" section
5) Parse the resulting HTML
You thing this is possible with your tecnology?
Thank you


"Phil Ide" <phil@idep.org.uk> ha scritto nel messaggio 
news:o6bed0vfr2xi.dlg@idep.org.uk...
> darkeagle,
>
>> I've sent a private mail.
>> Please let me know If you do not received.
>
> I have received - just knocking up a demo that will help you.
>
> Regards,
> -- 
> Phil Ide
>
> ***************************************
> * Xbase++ FAQ, Libraries and Sources: *
> * goto: http://www.idep.org.uk/xbase  *
> ***************************************
>
> I will not hold lavish banquets in the middle of a famine. The good PR 
> among
> the guests doesn't make up for the bad PR among the masses.
>          [Things I'd do as an Evil Overlord]
Phil Ide
Re: Automated interaction with a web site: possible?
on Mon, 13 Feb 2006 17:11:47 +0000
darkeagle,

> As a sample, my xbase application can do a thing like this (all in batch 
> mode):
> 1) connect to www.ebay.it
> 2) navigate to the login page of ebay,
> 3) insert username and password and submitting form
> 4) Navigate to "myEbay" section
> 5) Parse the resulting HTML
> You thing this is possible with your tecnology?

I've replied by private email and also in the Xb2.net NG, but yes, very
easily.  Submit a simple URL to login (no need to fetch the login page
first), which will return a cookie.  Then submit the myEbay url and extract
the data you want.

Regards,

Phil Ide

***************************************
* Xbase++ FAQ, Libraries and Sources: *
* goto: http://www.idep.org.uk/xbase  *
***************************************

I will not fly into a rage and kill a messenger who brings me bad news just
to illustrate how evil I really am. Good messengers are hard to come by.
          [Things I'd do as an Evil Overlord]