Scraping JS-fueled webpages with python 3.x on windows computer
<p>This is my first post here, so I hope you'll be kind enough to point out my mistakes if ever I crossed any rules of this website.</p>

<p>First off, I'm quite "self-taught" in both english and python, so I apologize in advance if I make any language mistakes.</p>

<p>So, I'm learning Python as I said, and I was trying to write a script able to scrape a webpage to get an element of it so that it continues to the next link, and so on. On my different attempts, I sometimes stumbled on a webpage whose interesting link is generated by a script (most certainly JavaScript), and so, when the webpage is retrieved by requests.get(url) doesn't contain the link I'm interested in (while I see it in my web browser while Inspecting the page or viewing source code.</p>

<p>I KNOW there is the Selenium solution, but I was wondering if there was ANOTHER way. I found several, but none I actually got to make work. I've tried with dryscrape, which I found out, isn't supported on Windows computers.</p>

<p>Any hint on what direction I should direct my research at?
Again, I'm hoping for a solution without using selenium, that works on Windows computers.</p>

<p><strong>EDIT:</strong> Oh, seeing as the answers suggested that already, I probably should have mentionned that my code uses requests and BeautifulSoup already. Problem is, neither deals with javascript that modifies the source code directly in the client. When I try to scrape the webpage in question with BeautifulSoup, many tags (including the one I'm interested in) don't appear in the whole page. It appears JavaScript injects some code when the page is loaded within the browser. In any case, there is no occurence of the link I'm after in the webpage I point requests.get at, nor in the requests.get(url).text I am looking in with BS4.</p>

<p>Thanks folks :)</p>

<p>I would suggest you try <a href="https://www.crummy.com/software/BeautifulSoup/" rel="nofollow">Beautiful Soup</a></p>

javascript python