How JustAnswer Works:
  • Ask an Expert
    Experts are full of valuable knowledge and are ready to help with any question. Credentials confirmed by a Fortune 500 verification firm.
  • Get a Professional Answer
    Via email, text message, or notification as you wait on our site.
    Ask follow up questions if you need to.
  • 100% Satisfaction Guarantee
    Rate the answer you receive.
Ask Pete Your Own Question
Pete
Pete, Computer Engineer
Category: Programming
Satisfied Customers: 17319
Experience:  Over 16 years of experience in computer programming.
30970729
Type Your Programming Question Here...
Pete is online now
A new question is answered every 9 seconds

I use R . I'm trying to login to a webpage and read the

Customer Question

I use R for scraping. I'm trying to login to a webpage and read the content afterwards, but it's not successful. Here is the code:
destination <- "http://artsalesindex.artinfo.com/auctions/Lajos-A-1897312/Portrait-of-young-lady-with-feather-fan-1885"
result <- POST(destination,
endoce ="form",
body = list( lightboxusername="username", lightboxpassword="password")
)
piece_html <-read_html(result)
x % html_nodes(".style14Reg222") %>% html_text()
if (length(x)==0) {price2 <- "NA"} else{price2 <- x}
Submitted: 1 year ago.
Category: Programming
Customer: replied 1 year ago.
piece_html <-read_html(result)
x <- piece_html %>% html_nodes(".style14Reg222") %>% html_text()
if (length(x)==0) {price2 <- "NA"} else{price2 <- x}is actually correct
Expert:  Pete replied 1 year ago.

Hi there,

Did you get the correct code sorted?

Customer: replied 1 year ago.
There was a typo in the first question I sent, here is the code I have, but it still does not work.
destination <- "http://artsalesindex.artinfo.com/auctions/Lajos-A-1897312/Portrait-of-young-lady-with-feather-fan-1885"
result <- POST(destination,endoce ="form",
body = list( lightboxusername="username", lightboxpassword="password")
)
piece_html <-read_html(result)
x <- piece_html %>% html_nodes(".style14Reg222") %>% html_text()
if (length(x)==0) {price2 <- "NA"} else{price2 <- x}My goal is to read data fields such as PRICE SOLD and ORIGINAL CURRENCY OF SALE which show up after login.
Customer: replied 1 year ago.
I can give you my user and pass if it's necessary.
Expert:  Pete replied 1 year ago.

Is it reading, but reporting incorrect data, or not reading at all?

Customer: replied 1 year ago.
here is a complete description. For the exact same url I provided above, there are two page sources. One is before login, and the other after login.With the code I provided, I can read the data fields which is publicly available( such as title, year, medium, ...), but I cannot read those those that show up in the page source after login ( such as PRICE SOLD). I think POST command should somehow post the login form required to the website, and get back the response, and the response should be the page source after login; but apparently it doesn't. It should be very easy to check with Rstudio.
For example I can easily read title field with these line, since it is in the page source before login.
x <- piece_html %>% html_nodes(".style16Italic222") %>% html_text()
if (length(x)==0) {title <- "NA"} else {title <- x}
but I cannot read the PRICE SOLD and ORIGINAL CURRENCY OF SALE.The result is "PRICE SOLD\n" , and NA respectively. SO I guess it does not read the page source after login.
Customer: replied 1 year ago.
html nodes after login should be .style14Reg222 for price, and .style13Reg222 for original currency of sale.
Expert:  Pete replied 1 year ago.

Are you only interested in the one site, or have you tried the code with any other sites?

Customer: replied 1 year ago.
I'm interested in just this website.
Expert:  Pete replied 1 year ago.

I can't see anything wrong with the code, so I'm not sure why it's not working with this website.

I'll check with the other experts to see if they have any ideas...

Customer: replied 1 year ago.
The issue is logging in. The part which reads the nodes should be fine, and it can read different publicly available fields. But for reading I have to give url plus credentials, which credentials are posted by POST command, which I guess does not do the job. let me know if you need credentials so you can check the page sources before and after login.
Customer: replied 1 year ago.
could it because there are several forms in the webpage? but I don't know how to target just one form .

Related Programming Questions