Download specific file in url using PHP/Python
<p>I previously used to use <code>wget -r</code> on the linux terminal for downloading files with certain extensions:</p>

<pre><code>wget -r -A Ext URL
</code></pre>

<p>But now I was assigned by my lecturer to do the same thing using PHP or Python.  Who can help?</p>

<p>For Python, use a web-crawler library such as scrapy.</p>

<p>It has <a href="https://doc.scrapy.org/en/latest/topics/spiders.html" rel="nofollow">classes</a> that do all the work when passed arguments similar to those you put on the <code>wget</code> command line.</p>

<p>You can use scrapy <a href="https://doc.scrapy.org/en/latest/topics/media-pipeline.html" rel="nofollow">pipelines</a> to filter out unwanted downloads, and value-add the downloads such as adding thumbnails.</p>

php python web-crawler wget