Python using logging module than print
<p>I want to use a logging module to replace the <code>print()</code>, Would appreciate some suggestions on how o get this done.</p>

<p><strong>Code in Context</strong></p>

<pre><code>from bs4 import BeautifulSoup, Tag
import requests
from pprint import pprint
import sys
import logging
from logging.config import fileConfig

fileConfig("logging.conf")

url = "http://hortonworks.com/careers/open-positions/"

response = requests.get(url)

if response.status_code != 200:
    print("Request failed with http code {}".format(response.status_code))
    sys.exit(1)

soup = BeautifulSoup(response.text, "html.parser")

jobs = []
div_main = soup.select("div#careers_list")

for div in div_main:
    for element in div:
        if isinstance(element, Tag) and "class" in element.attrs:
            if "department_title" in element.attrs["class"]:
                department_title = element.get_text().strip()
            elif "career" in element.attrs["class"]:
                location = element.select("div.location")[0].get_text().strip()
                title = element.select("div.title")[0].get_text().strip()
                job = {
                    "job_location": location,
                    "job_title": title,
                    "job_dept": department_title
                }
                jobs.append(job)

logging.info(jobs)
</code></pre>

<p>I  replaced the <code>pprint</code> with the logging.info.</p>

<p>my logging.conf file</p>

<pre><code>[loggers]
keys=root

[handlers]
keys=hand01

[formatters]
keys=form01

[logger_root]
level=DEBUG
handlers=hand01

[handler_hand01]
level=DEBUG
class=StreamHandler
args=(sys.stdout,)
formatter=form01

[formatter_form01]
format= %(processName)s %(asctime)s %(pathname)s %(levelname)-9s %(message)s
datefmt=%Y-%m-%d %H:%M:%S
class=logging.Formatter
</code></pre>

<p>Is this the right approach?</p>

<p>What you want to achieve is still not quite clear to me - specially, why you want to use a logger here. It looks like your script's goal is to output the jobs listing ? If yes, printing it to stdout was probably the RightThing(tm) to do. </p>

<p>As a general rule: for a command line application, stdout is for normal program output, stderr for error reporting / debug trace stuff. </p>

<p>Logging is usually used for complex application (GUI, daemons etc), and by well written library code, with the library code <em>only</em> logging, and leaving the logging configuration part to the application (because the library code cannot know in which context it will be used), and it is used for reporting (errors, warnings, debug trace...) so the admins and or developers can analyse the application's behaviour and eventually trace problems when something goes wrong. </p>

<p>IOW, using logging in a =~ 40 lines quick script with not even a function in it really doesn't make sense, and using it to send the script's output to stdout is actually a wtf (please pardon my french). </p>

<p>Now if what you wanted was insight on "how" to use the logging module that's another question. </p>

<p>First, as I stated above, one of the points of the <code>logging</code> module is to decouple logging configuration (which should be done by the application's entry point, as soon as possible, and can be done in anyway that please the application's developper - using <code>basicConfig</code>, <code>dictConfig</code> or plain manual configuration) from logging use (which happens inside the application's and libraries code). Typically this means you have a very simple main script (the "application entry point") that merely set up logging etc, checks environment variables / command line arguments etc, imports the application code (which itself will import required libs) and run the application.</p>

<p>In this context, each module (either application modules or library module) will ask for it's own logger, canonically using the module's name (ie <code>myapp.mypackage.mymodule</code>) as provided by the <code>__name__</code> magic variable as logger name. This will build a "tree" of logger (with, from root to leaf, 'root' -> 'myapp' -> 'mypackage' -> 'mymodule'), each of which can be configured specifically or relie on it's parent logger config. Then, it's up to you as the module's developper to log what you think will be relevant, using the appropriate level. </p>

python logging beautifulsoup python-requests