on 06-17-2008 8:18 PM
Greetings all,
I am using a HTTP Post Action in BLS - Vs 11.5.
The objective is to return a HTTP string from a jsp page, so I can scrap the page.
The URL: http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=2008-06-16 (obviously you might have to change the date to Yesterdays Date)
I have set up the HTTP Post Action with
URL Address of: http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp
Post Document: marketdate=2008-06-16
Encoding: UTF-8
ID and Password are provided
Content Type: application/x-www-form-urlencoded
Post_0.ReturnAsString comes back with typical HTML content for Today's Date.
Testing has shown that on a fresh IE window. Using the above url will show today's data.
Hitting this url again the second time will get the browser to yesterday's data.
So any suggestions on how to make this happen.
(btw, I have no control over the external public ERCOT website so changing thier jsp page isn't an option.)
Anyway we could emulate that action in BLS - v11.5?
Thanks
Dennis
Dennis:
Why not just use the HTML Loader? That would work perfectly, wouldn't it?
Rick
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I was unable to replicate your issue but....
The issue you have may be that the Post URL parameter isn't set dynamically, use this expression in the link editor:
"http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=" & datefromxmlformat( datenow, "yyyy-MM-dd")
Sam
PS: Change the content type to text/html...
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Greetings All,
My goal on this project is to obtain a scraping of the page for yesterday.
It seems no matter what I place in the post Document location, It only returns todays data.
I explained that if I was doing this manually from a new IE browser it has to update twice with the given url string including the marketdate=yyyy-mm-dd
Format of the url is not the problem, the problem seems to be the required repost of the url string???
Hope that makes sense.
Dennis
Dennis,
I see the issue now...sorry about that. Use the Logic -> ForLoop repeater action and set it to repeat twice and call the POST twice? Then use this expression in the link URL:
"http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=" & datefromxmlformat( dateadddays(datenow, -1), "yyyy-MM-dd")
Sam
It's not working for me???
Did that work for you?
I have attached my sample... It's a repeater, a http Post and a tracer.
<?xml version="1.0" encoding="UTF-8"?>
<Transaction xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<TransactionAttributes>
<ContextItem>
<Name>Description</Name>
<Description>Transaction Description</Description>
<MinRange>0</MinRange>
<MaxRange>0</MaxRange>
<Value xsi:type="xsd:string"/>
<ReadOnly>true</ReadOnly>
</ContextItem>
</TransactionAttributes>
<Name>west_test/html Post Question</Name>
<Version>1</Version>
<WriterRoles/>
<ReaderRoles/>
<Context/>
<Local/>
<Layout>
<GUILayoutItem>
<Name>Sequence</Name>
<X>8</X>
<Y>8</Y>
<Width>120</Width>
<SpacingWidth>120</SpacingWidth>
<DescendantWidth>120</DescendantWidth>
<Height>96</Height>
</GUILayoutItem>
<GUILayoutItem>
<Name>Post_0</Name>
<X>16</X>
<Y>24</Y>
<Width>104</Width>
<SpacingWidth>0</SpacingWidth>
<DescendantWidth>0</DescendantWidth>
<Height>72</Height>
</GUILayoutItem>
<GUILayoutItem>
<Name>Sequence_0</Name>
<X>8</X>
<Y>136</Y>
<Width>120</Width>
<SpacingWidth>120</SpacingWidth>
<DescendantWidth>0</DescendantWidth>
<Height>96</Height>
</GUILayoutItem>
<GUILayoutItem>
<Name>Tracer_0</Name>
<X>16</X>
<Y>152</Y>
<Width>104</Width>
<SpacingWidth>0</SpacingWidth>
<DescendantWidth>0</DescendantWidth>
<Height>72</Height>
</GUILayoutItem>
</Layout>
<Actions>
<ContextItem>
<Name>Post_0</Name>
<Description/>
<MinRange>0</MinRange>
<MaxRange>0</MaxRange>
<Value xsi:type="Post">
<URL><a href="http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=2008-06-18" TARGET="test_blank">http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=2008-06-18</a></URL>
<Encoding>UTF-8</Encoding>
<ContentType>text/html</ContentType>
<Timeout>60</Timeout>
<UserName>YOUSUPPLY</UserName>
<PostData/>
<UserPassword>YOUROWN</UserPassword>
<ReturnAsString/>
<ReturnAsXML>
<ReferenceDocumentSource/>
</ReturnAsXML>
</Value>
<ReadOnly>false</ReadOnly>
</ContextItem>
<ContextItem>
<Name>Tracer_0</Name>
<Description/>
<MinRange>0</MinRange>
<MaxRange>0</MaxRange>
<Value xsi:type="Tracer">
<Level>INFO</Level>
<Message/>
</Value>
<ReadOnly>false</ReadOnly>
</ContextItem>
</Actions>
<Steps>
<Step xsi:type="ActionSequence">
<Name>Sequence</Name>
<Description/>
<Steps>
<Step xsi:type="ActionSequence">
<Name>Sequence_0</Name>
<Description/>
<Steps/>
<Actions>
<Action>
<Name>Tracer_0</Name>
<Description/>
<IncomingLinks>
<Assign xsi:type="Assign">
<Name/>
<Description/>
<To>Tracer_0.Message</To>
<From>Post_0.ReturnAsString</From>
</Assign>
</IncomingLinks>
<OutgoingLinks/>
</Action>
</Actions>
</Step>
</Steps>
<Actions>
<Action>
<Name>Post_0</Name>
<Description/>
<IncomingLinks>
<Assign xsi:type="Assign">
<Name/>
<Description/>
<To>Post_0.URL</To>
<From>"http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=" & datefromxmlformat( datenow, "yyyy-MM-dd")"</From>
</Assign>
</IncomingLinks>
<OutgoingLinks/>
</Action>
</Actions>
</Step>
</Steps>
</Transaction>
User | Count |
---|---|
9 | |
7 | |
3 | |
2 | |
2 | |
2 | |
1 | |
1 | |
1 | |
1 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.