BeautifulSoup use unique CSS Selector
<p>From this <a href="https://www.placetel.de/status" rel="nofollow">page</a>, I need to get the status from "Anbindung an das Telefonnetz". </p>

<p>I identified 2 ways to get it: </p>

<ol>
<li>If the status contains the sentence "Das System arbeitet einwandfrei";</li>
<li>If the color of the background is green. </li>
</ol>

<p>I have chosen to go with the first option. </p>

<p>I use Python/BeautifulSoup to scrape the page. The thing is, there is no unique id/class or whatsoever to get this element. <br>
I then decided to use the CSS selector of this particular element, which is the following: </p>

<pre><code>div.system-item:nth-child(2) &gt; div:nth-child(1) &gt; p:nth-child(3)
</code></pre>

<p>And use it like this: </p>

<pre><code>print(page.select("div.system-item:nth-child(2) &gt; div:nth-child(1) &gt; p:nth-child(3)"))
</code></pre>

<p>However, the only thing I get is an empty element (<code>[]</code>). </p>

<p>What could I try more to get this particular element? </p>

<p><strong>EDIT</strong> <br>
As some of you recommend it, here is the un-complete HTML source of the page.<br>. But to be practical, i recommend you to just take a look yourself at the <a href="https://www.placetel.de/status" rel="nofollow">page</a></p>

<pre><code>&lt;!doctype html&gt;
&lt;head&gt;
    &lt;meta charset="utf-8"&gt;

            &lt;title&gt;Aktueller Status | Placetel&lt;/title&gt;

    &lt;meta http-equiv="X-UA-Compatible" content="IE=Edge"&gt;
    &lt;meta name="msvalidate.01" content="756F6E40DD887A659CE83E5A92FFBB62"&gt;
    &lt;meta name="viewport" content="width=device-width, initial-scale=1.0"&gt;

    &lt;meta name="generator" content="Kirby 2.3.2"&gt;

    &lt;meta name="description" content="Placetel Systemstatus: Erfahren Sie mehr &amp;uuml;ber den aktuellen Status der Placetel Telefonanlage."&gt;
    &lt;meta name="keywords" content=""&gt;

        &lt;meta name="robots" content="index,follow,noodp,noydir"&gt;

    &lt;link rel="canonical" href="https://www.placetel.de/status"&gt;
    &lt;link rel="publisher" href="https://plus.google.com/b/111027512373770716962/111027512373770716962/posts"&gt;

    &lt;link rel="shortcut icon" href="/favicon.ico"&gt;
    &lt;link rel="apple-touch-icon" href="/apple-touch-icon.png"&gt;
    &lt;meta name="msapplication-TileColor" content="#0e70b9"&gt;
    &lt;meta name="msapplication-TileImage" content="/ms-tile-icon.png"&gt;
    &lt;meta name="theme-color" content="#0e70b9"&gt;

    &lt;script src="//use.typekit.net/rnw8lad.js"&gt;&lt;/script&gt;
    &lt;script&gt;try { Typekit.load({ async: true }); } catch (e) {}&lt;/script&gt;

    &lt;link rel="stylesheet" href="https://www.placetel.de/assets/dist/css/main.css"&gt;    &lt;script src="https://www.placetel.de/assets/dist/js/modernizr.js"&gt;&lt;/script&gt;
    &lt;link rel="dns-prefetch" href="//app.marketizator.com"/&gt;
    &lt;script&gt;
        var _mktz = _mktz || [];
        _mktz.cc_domain = 'placetel.de';
    &lt;/script&gt;
    &lt;script type="text/javascript" src="//d2tgfbvjf3q6hn.cloudfront.net/js/o17fe41.js"&gt;&lt;/script&gt;
&lt;/head&gt;
&lt;body id="í ½íº" class="page page-template-page-sections page-uid-status"&gt;

&lt;script&gt;
    var gaProperty = 'UA-17631409-3';
    var disableStr = 'ga-disable-' + gaProperty;
    if (document.cookie.indexOf(disableStr + '=true') &gt; -1) {
        window[disableStr] = true;
    }
    function gaOptout() {
        document.cookie = disableStr + '=true; expires=Thu, 31 Dec 2099 23:59:59 UTC; path=/';
        window[disableStr] = true;
    }
&lt;/script&gt;

&lt;!-- Google Tag Manager --&gt;
&lt;noscript&gt;&lt;iframe src="//www.googletagmanager.com/ns.html?id=GTM-KDNGCC"
                  height="0" width="0" style="display:none;visibility:hidden"&gt;&lt;/iframe&gt;&lt;/noscript&gt;
&lt;script&gt;(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
        new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
                                                  j=d.createElement(s),dl=l!='dataLayer'?'&amp;l='+l:'';j.async=true;j.src=
        '//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
    })(window,document,'script','dataLayer','GTM-KDNGCC');&lt;/script&gt;
&lt;!-- End Google Tag Manager --&gt;
&lt;header class="header header-condensed" id="header"&gt;
    &lt;div class="container-fluid"&gt;

&lt;nav class="navigation navigation-top"&gt;
    &lt;ul&gt;
                    &lt;li class=" "&gt;
                &lt;a title="Unternehmen" href="https://www.placetel.de/unternehmen"&gt;

                    &lt;span&gt;Unternehmen&lt;/span&gt;
                &lt;/a&gt;
            &lt;/li&gt;
                    &lt;li class=" "&gt;
                &lt;a title="Partner werden" href="https://www.placetel.de/partner"&gt;

                    &lt;span&gt;Partner werden&lt;/span&gt;
                &lt;/a&gt;
            &lt;/li&gt;
                    &lt;li class=" "&gt;
                &lt;a title="Support" href="https://www.placetel.de/support"&gt;

                    &lt;span&gt;Support&lt;/span&gt;
                &lt;/a&gt;
            &lt;/li&gt;
                    &lt;li class=" "&gt;
                &lt;a title="Suche" href="javascript:modal('search')"&gt;

                    &lt;span&gt;Suche&lt;/span&gt;
                &lt;/a&gt;
            &lt;/li&gt;
                &lt;li class="navigation-top-support"&gt;
            &lt;a href="https://www.placetel.de/support"&gt;
                &lt;svg class="svg-phone"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-phone"&gt;&lt;/use&gt;&lt;/svg&gt;                &lt;span&gt;0221 29 191 999&lt;/span&gt;
            &lt;/a&gt;
        &lt;/li&gt;
        &lt;li class="navigation-top-login"&gt;
            &lt;a href="https://app.placetel.de/account/login"&gt;
                &lt;span&gt;Login&lt;/span&gt;
            &lt;/a&gt;
        &lt;/li&gt;
    &lt;/ul&gt;
&lt;/nav&gt;    &lt;/div&gt;

    &lt;div class="container-fluid"&gt;
        &lt;a class="site-logo" href="https://www.placetel.de"&gt;
            &lt;svg class="svg-placetel-logo"&gt;&lt;title&gt;Placetel&lt;/title&gt; &lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-placetel-logo"&gt;&lt;/use&gt;&lt;/svg&gt;        &lt;/a&gt;

&lt;nav class="navigation navigation-main" id="navigation-main"&gt;
    &lt;ul&gt;

            &lt;li class="has-sub-navigation"&gt;
                &lt;a title="Telefonanlage" href="https://www.placetel.de/telefonanlage"
                   class=""&gt;
                    &lt;span&gt;Telefonanlage&lt;/span&gt;

                                            &lt;svg class="svg-arrow"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-arrow"&gt;&lt;/use&gt;&lt;/svg&gt;                                    &lt;/a&gt;

                                    &lt;nav class="sub-navigation"&gt;
                        &lt;ul&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/telefonanlage"&gt;
                                        Vorteile                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/telefonanlage/preise"&gt;
                                        Preise                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/telefonanlage/funktionen"&gt;
                                        Funktionen                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/telefonanlage/unified-communication"&gt;
                                        Unified Communication                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/telefonanlage/funktionsweise"&gt;
                                        Wie funktioniert es?                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/telefonanlage/isdn-abschaltung"&gt;
                                        ISDN-Abschaltung                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/telefonanlage/faq"&gt;
                                        FAQ                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                    &lt;/ul&gt;
                    &lt;/nav&gt;
                            &lt;/li&gt;

            &lt;li class=""&gt;
                &lt;a title="Trunking" href="https://www.placetel.de/sip-trunking"
                   class=""&gt;
                    &lt;span&gt;Trunking&lt;/span&gt;

                                    &lt;/a&gt;

                            &lt;/li&gt;

            &lt;li class=""&gt;
                &lt;a title="Mobilfunk" href="https://www.placetel.de/mobilfunk"
                   class=""&gt;
                    &lt;span&gt;Mobilfunk&lt;/span&gt;

                                    &lt;/a&gt;

                            &lt;/li&gt;

            &lt;li class="navigation-main-shop"&gt;
                &lt;a title="Endger&amp;auml;te-Shop" href="/shop/"
                   class=""&gt;
                    &lt;span&gt;Endger&amp;auml;te-Shop&lt;/span&gt;

                                    &lt;/a&gt;

                            &lt;/li&gt;

            &lt;li class="visible-xs-block visible-sm-block"&gt;
                &lt;a title="Support" href="https://www.placetel.de/support"
                   class=""&gt;
                    &lt;span&gt;Support&lt;/span&gt;

                                    &lt;/a&gt;

                            &lt;/li&gt;

            &lt;li class="visible-xs-block visible-sm-block"&gt;
                &lt;a title="Partner" href="https://www.placetel.de/partner"
                   class=""&gt;
                    &lt;span&gt;Partner&lt;/span&gt;

                                    &lt;/a&gt;

                            &lt;/li&gt;

            &lt;li class="has-sub-navigation visible-xs-block visible-sm-block"&gt;
                &lt;a title="Unternehmen" href="https://www.placetel.de/unternehmen"
                   class=""&gt;
                    &lt;span&gt;Unternehmen&lt;/span&gt;

                                            &lt;svg class="svg-arrow"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-arrow"&gt;&lt;/use&gt;&lt;/svg&gt;                                    &lt;/a&gt;

                                    &lt;nav class="sub-navigation"&gt;
                        &lt;ul&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/unternehmen"&gt;
                                        &amp;Uuml;ber uns                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/unternehmen/technologie"&gt;
                                        Technologie                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/unternehmen/jobs"&gt;
                                        Jobs                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/unternehmen/events"&gt;
                                        Events                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/unternehmen/presse"&gt;
                                        Presse                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                            &lt;li class=""&gt;
                                    &lt;a href="https://www.placetel.de/unternehmen/kontakt"&gt;
                                        Kontakt                                    &lt;/a&gt;
                                &lt;/li&gt;
                                                    &lt;/ul&gt;
                    &lt;/nav&gt;
                            &lt;/li&gt;

            &lt;li class="navigation-main-register"&gt;
                &lt;a title="Kostenlos testen!" href="javascript:modal('register')"
                   class="btn"&gt;
                    &lt;span&gt;Kostenlos testen!&lt;/span&gt;

                                    &lt;/a&gt;

                            &lt;/li&gt;
            &lt;/ul&gt;
&lt;/nav&gt;        
        &lt;a class="site-navigation-toggle" id="hotdog"&gt;
            &lt;i&gt;
                &lt;span&gt;&lt;/span&gt;
            &lt;/i&gt; MenÃ¼
        &lt;/a&gt;
    &lt;/div&gt;
&lt;/header&gt;


            &lt;section class="section section-full section-full-section-einleitung-text section-full-normal"&gt;
    &lt;div class="container-fluid typography typography-dark"&gt;
                    &lt;h2 class="section-full-title"&gt;Der Placetel System Status&lt;/h2&gt;

                    &lt;h3 class="section-full-subtitle"&gt;Jeden Tag einen Grund zur Freude.&lt;/h3&gt;

                    &lt;p&gt;Wir bei Placetel haben ein Lieblingswort: âlÃ¤uftâ. Der Grund: Ihre Placetel Telefonanlage funktioniert nÃ¤mlich immer. DarÃ¼ber freuen wir uns natÃ¼rlich riesig. Da aber erst eine geteilte Freude eine richtige Freude ist, haben wir Ihnen diese Statusseite eingerichtet.  Diese Seite informiert Sie jeden Tag Ã¼ber den einwandfreien Status Ihrer Anlage.&lt;br /&gt;
Und falls etwas mal nicht so perfekt funktionieren sollte wie gewohnt, kÃ¶nnen Sie uns den Fehler gern  melden.&lt;/p&gt;        
            &lt;/div&gt;

            &lt;style&gt;
            .section-full-section-einleitung-text {
                background-color: ;
            }
        &lt;/style&gt;

    &lt;/section&gt;    

            &lt;section class="section section-system"&gt;
    &lt;a class="btn btn-primary btn-transparent btn-with-icon" href="javascript:location.reload();"&gt;
        &lt;svg class="svg-refresh"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-refresh"&gt;&lt;/use&gt;&lt;/svg&gt;        Status aktualisieren
    &lt;/a&gt;

    &lt;div class="system flex-grid typography typography-light"&gt;
        &lt;div class="system-item system-item-green flex-grid-item"&gt;
            &lt;div class="system-item-inner"&gt;
                &lt;h6&gt;
                    System                &lt;/h6&gt;

                &lt;i&gt;
                    &lt;svg class="svg-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-included"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-dots"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-dots"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-not-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-not-included"&gt;&lt;/use&gt;&lt;/svg&gt;                &lt;/i&gt;

                &lt;p&gt;
                    Das System arbeitet einwandfrei&lt;br&gt;
                    11:10 Uhr
                &lt;/p&gt;

                            &lt;/div&gt;
        &lt;/div&gt;

        &lt;div class="system-item system-item-green flex-grid-item"&gt;
            &lt;div class="system-item-inner"&gt;
                &lt;h6&gt;
                    Anbindung an das  Telefonnetz                &lt;/h6&gt;

                &lt;i&gt;
                    &lt;svg class="svg-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-included"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-dots"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-dots"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-not-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-not-included"&gt;&lt;/use&gt;&lt;/svg&gt;                &lt;/i&gt;

                &lt;p&gt;
                    Das System arbeitet einwandfrei&lt;br&gt;
                    11:10 Uhr
                &lt;/p&gt;

                            &lt;/div&gt;
        &lt;/div&gt;

        &lt;div class="system-item system-item-green flex-grid-item"&gt;
            &lt;div class="system-item-inner"&gt;
                &lt;h6&gt;
                    Faxsystem                &lt;/h6&gt;

                &lt;i&gt;
                    &lt;svg class="svg-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-included"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-dots"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-dots"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-not-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-not-included"&gt;&lt;/use&gt;&lt;/svg&gt;                &lt;/i&gt;

                &lt;p&gt;
                    Das System arbeitet einwandfrei&lt;br&gt;
                    11:10 Uhr
                &lt;/p&gt;

                            &lt;/div&gt;
        &lt;/div&gt;

        &lt;div class="system-item system-item-green flex-grid-item"&gt;
            &lt;div class="system-item-inner"&gt;
                &lt;h6&gt;
                    Konferenzsystem                &lt;/h6&gt;

                &lt;i&gt;
                    &lt;svg class="svg-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-included"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-dots"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-dots"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-not-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-not-included"&gt;&lt;/use&gt;&lt;/svg&gt;                &lt;/i&gt;

                &lt;p&gt;
                    Das System arbeitet einwandfrei&lt;br&gt;
                    11:10 Uhr
                &lt;/p&gt;

                            &lt;/div&gt;
        &lt;/div&gt;

        &lt;div class="system-item system-item-green flex-grid-item"&gt;
            &lt;div class="system-item-inner"&gt;
                &lt;h6&gt;
                    Features und Optionen                &lt;/h6&gt;

                &lt;i&gt;
                    &lt;svg class="svg-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-included"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-dots"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-dots"&gt;&lt;/use&gt;&lt;/svg&gt;                    &lt;svg class="svg-not-included"&gt;&lt;use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.placetel.de/assets/dist/sprites/svg/sprite.1471515912.svg#svg-not-included"&gt;&lt;/use&gt;&lt;/svg&gt;                &lt;/i&gt;

                &lt;p&gt;
                    Das System arbeitet einwandfrei&lt;br&gt;
                    11:10 Uhr
                &lt;/p&gt;

                            &lt;/div&gt;
        &lt;/div&gt;
    &lt;/div&gt;
&lt;/section&gt;    

&lt;/body&gt;
&lt;/html&gt;
</code></pre>

<p>You can use the text to find the h6 for <code>Anbindung an das  Telefonnetz</code> and get the p sibling:</p>

<pre><code>import requests
import re
r = requests.get("https://www.placetel.de/status").content
soup = BeautifulSoup(r, "lxml")

h6 = soup.find("h6", text=re.compile(ur"Anbindung an das  Telefonnetz", re.I))
if h6:
    print(h6.find_next_sibling("p"))
</code></pre>

<p>If you want full css3 selector support you can use <em>lxml's</em> <a href="http://lxml.de/cssselect.html" rel="nofollow">cssselect</a>:</p>

<pre><code>from lxml import html
tree = html.fromstring(r)
print(tree.cssselect("div.system-item:nth-child(2) &gt; div:nth-child(1) &gt; p:nth-child(3)")
</code></pre>

<p>You can also just search by the text so if the h6 turned to a h5 or any other tag it would make no odds:</p>

<pre><code>match = soup.find(text=re.compile(ur"Anbindung an das  Telefonnetz", re.I))

if match:
    print(match.parent.find_next_sibling("p").text)
</code></pre>

<p>You could use the outer div to localise the search for the text, bs4 is pretty flexible. Just selecting all the <code>div.system-item</code> and indexing  would break if the order changed and you would not know as there would be no error so looking for the text is probably actually a safer approach.</p>

python html css beautifulsoup