Getting all HTML from requests.get()
<p>I just started with web scraping with Python and hit the wall.
I am using the requests library to get the HTML code from a website. For example, the Google search result website: "<a href="https://www.google.com/?gws_rd=ssl#q=ball" rel="nofollow">https://www.google.com/?gws_rd=ssl#q=ball</a>"</p>

<p>When I hit <kbd>F12</kbd> and check the HTML, it looks different than with:</p>

<pre><code>site = requests.get("https://www.google.com/?gws_rd=ssl#q=ball")
print(site.text)
</code></pre>

<p>with <code>requests.get</code>, text is much shorter and not all information is visible (it starts with <code>!doctype</code>, however).
Because of that I am unable to work with this HTML.</p>

<p>Can you tell me where the mistake is?</p>

<hr>

<p>This is actually an exercise from the book "Automate the boring stuff with Python". The task is to search for some item Google and then find few first results with HTML locators. I cannot do it because when I use <code>requests.get()</code> I cannot see any objects for links in the HTML code.</p>

<p>Some HTML elements are generated by JavaScript.</p>

<p>Use "show source code" from your browser to see the original code. It must be similar to the Request response text. </p>

python html python-requests