cancel
Showing results for 
Search instead for 
Did you mean: 

HTTP Post Action Block

Former Member
0 Kudos

Greetings all,

I am using a HTTP Post Action in BLS - Vs 11.5.

The objective is to return a HTTP string from a jsp page, so I can scrap the page.

The URL: http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=2008-06-16 (obviously you might have to change the date to Yesterdays Date)

I have set up the HTTP Post Action with

URL Address of: http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp

Post Document: marketdate=2008-06-16

Encoding: UTF-8

ID and Password are provided

Content Type: application/x-www-form-urlencoded

Post_0.ReturnAsString comes back with typical HTML content for Today's Date.

Testing has shown that on a fresh IE window. Using the above url will show today's data.

Hitting this url again the second time will get the browser to yesterday's data.

So any suggestions on how to make this happen.

(btw, I have no control over the external public ERCOT website so changing thier jsp page isn't an option.)

Anyway we could emulate that action in BLS - v11.5?

Thanks

Dennis

Accepted Solutions (0)

Answers (2)

Answers (2)

Former Member
0 Kudos

Dennis:

Why not just use the HTML Loader? That would work perfectly, wouldn't it?

Rick

Former Member
0 Kudos

I started with the HTML Loader but it also seems not to do the Refresh type action required by the page to get to Yesterday's data.

0 Kudos

I was unable to replicate your issue but....

The issue you have may be that the Post URL parameter isn't set dynamically, use this expression in the link editor:

"http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=" & datefromxmlformat( datenow, "yyyy-MM-dd")

Sam

PS: Change the content type to text/html...

Former Member
0 Kudos

Greetings All,

My goal on this project is to obtain a scraping of the page for yesterday.

It seems no matter what I place in the post Document location, It only returns todays data.

I explained that if I was doing this manually from a new IE browser it has to update twice with the given url string including the marketdate=yyyy-mm-dd

Format of the url is not the problem, the problem seems to be the required repost of the url string???

Hope that makes sense.

Dennis

0 Kudos

Dennis,

I see the issue now...sorry about that. Use the Logic -> ForLoop repeater action and set it to repeat twice and call the POST twice? Then use this expression in the link URL:

"http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=" & datefromxmlformat( dateadddays(datenow, -1), "yyyy-MM-dd")

Sam

0 Kudos

In addition to that I would complain to the company who owns the site about the issue with their page.

Sam

Former Member
0 Kudos

It's not working for me???

Did that work for you?

I have attached my sample... It's a repeater, a http Post and a tracer.


<?xml version="1.0" encoding="UTF-8"?>
<Transaction xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<TransactionAttributes>
		<ContextItem>
			<Name>Description</Name>
			<Description>Transaction Description</Description>
			<MinRange>0</MinRange>
			<MaxRange>0</MaxRange>
			<Value xsi:type="xsd:string"/>
			<ReadOnly>true</ReadOnly>
		</ContextItem>
	</TransactionAttributes>
	<Name>west_test/html Post Question</Name>
	<Version>1</Version>
	<WriterRoles/>
	<ReaderRoles/>
	<Context/>
	<Local/>
	<Layout>
		<GUILayoutItem>
			<Name>Sequence</Name>
			<X>8</X>
			<Y>8</Y>
			<Width>120</Width>
			<SpacingWidth>120</SpacingWidth>
			<DescendantWidth>120</DescendantWidth>
			<Height>96</Height>
		</GUILayoutItem>
		<GUILayoutItem>
			<Name>Post_0</Name>
			<X>16</X>
			<Y>24</Y>
			<Width>104</Width>
			<SpacingWidth>0</SpacingWidth>
			<DescendantWidth>0</DescendantWidth>
			<Height>72</Height>
		</GUILayoutItem>
		<GUILayoutItem>
			<Name>Sequence_0</Name>
			<X>8</X>
			<Y>136</Y>
			<Width>120</Width>
			<SpacingWidth>120</SpacingWidth>
			<DescendantWidth>0</DescendantWidth>
			<Height>96</Height>
		</GUILayoutItem>
		<GUILayoutItem>
			<Name>Tracer_0</Name>
			<X>16</X>
			<Y>152</Y>
			<Width>104</Width>
			<SpacingWidth>0</SpacingWidth>
			<DescendantWidth>0</DescendantWidth>
			<Height>72</Height>
		</GUILayoutItem>
	</Layout>
	<Actions>
		<ContextItem>
			<Name>Post_0</Name>
			<Description/>
			<MinRange>0</MinRange>
			<MaxRange>0</MaxRange>
			<Value xsi:type="Post">
				<URL><a href="http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=2008-06-18" TARGET="test_blank">http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=2008-06-18</a></URL>
				<Encoding>UTF-8</Encoding>
				<ContentType>text/html</ContentType>
				<Timeout>60</Timeout>
				<UserName>YOUSUPPLY</UserName>
				<PostData/>
				<UserPassword>YOUROWN</UserPassword>
				<ReturnAsString/>
				<ReturnAsXML>
					<ReferenceDocumentSource/>
				</ReturnAsXML>
			</Value>
			<ReadOnly>false</ReadOnly>
		</ContextItem>
		<ContextItem>
			<Name>Tracer_0</Name>
			<Description/>
			<MinRange>0</MinRange>
			<MaxRange>0</MaxRange>
			<Value xsi:type="Tracer">
				<Level>INFO</Level>
				<Message/>
			</Value>
			<ReadOnly>false</ReadOnly>
		</ContextItem>
	</Actions>
	<Steps>
		<Step xsi:type="ActionSequence">
			<Name>Sequence</Name>
			<Description/>
			<Steps>
				<Step xsi:type="ActionSequence">
					<Name>Sequence_0</Name>
					<Description/>
					<Steps/>
					<Actions>
						<Action>
							<Name>Tracer_0</Name>
							<Description/>
							<IncomingLinks>
								<Assign xsi:type="Assign">
									<Name/>
									<Description/>
									<To>Tracer_0.Message</To>
									<From>Post_0.ReturnAsString</From>
								</Assign>
							</IncomingLinks>
							<OutgoingLinks/>
						</Action>
					</Actions>
				</Step>
			</Steps>
			<Actions>
				<Action>
					<Name>Post_0</Name>
					<Description/>
					<IncomingLinks>
						<Assign xsi:type="Assign">
							<Name/>
							<Description/>
							<To>Post_0.URL</To>
							<From>"http://mospublic.ercot.com/ercot/jsp/balancing_services_mcp.jsp?marketdate=" &amp; datefromxmlformat( datenow, "yyyy-MM-dd")"</From>
						</Assign>
					</IncomingLinks>
					<OutgoingLinks/>
				</Action>
			</Actions>
		</Step>
	</Steps>
</Transaction>