Python script for searching variable strings between two constant strings
<pre><code>import re

infile = open('document.txt','r')
outfile= open('output.txt','w')
copy = False
for line in infile:

    if line.strip() == "--operation():":
        bucket = []
        copy = True

    elif line.strip() == "StartOperation":
        for strings in bucket:
            outfile.write( strings + ',')
        for strings in bucket:
            outfile.write('\n')
        copy = False

    elif copy:
        bucket.append(line.strip()
</code></pre>

<p>CSV format is like this:</p>

<pre><code>id,          name,                poid,         error
5896, AutoAuthOSUserSubmit,     900105270,      0x4002
</code></pre>

<p>My log file has several sections starting with <code>==== START ====</code> and ending with <code>==== END   ====</code>.  I want to extract the string between <code>--operation():</code> and <code>StartOperation</code>. For example, <code>AutoAuthOSUserSubmit.</code> I also want to extract the <code>poid</code> value from line <code>poid: 900105270, poidLen: 9</code>. Finally, I want to extract the return value, e.g <code>0x4002</code> if <code>Roll back all updates</code> is found after it. </p>

<p>I am not even able to extract point the original text if <code>Start</code> and <code>End</code> are not on the same line. How do I go about doing that?</p>

<p>This is a sample LOG extract with two paragraphs:</p>

<pre><code>-- 08/24 02:07:56 [mds.ecas(5896) ECAS_CP1] **==== START ====**
open file /ecas/public/onsite-be/config/timer.conf failed
INFO 08/24/16 02:07:56  salt1be-d1-ap(**5896**/0)  main.c(780*****):--operation(): AutoAuthOSUserSubmit. StartOperation*****
INFO 08/24/16 02:07:56  salt1be-d1-ap(5896/0)  main.c(784):--Client Information: Request from host 'malt-d1-wb' process id 12382.
DEBUG 08/24/16 02:07:56  salt1be-d1-ap(5896/0)  TOci.cc(571):FetchServiceObjects: ServiceCert.sql
DEBUG 08/22/16 23:15:53  pepper1be-d1-ap(2680/0)  vsserviceagent.cpp(517):Generate Certificate 2: c1cd00d5c3de082360a08730fef9cd1d
DEBUG 08/22/16 23:15:53  pepper1be-d1-ap(2680/0)  junk.c(1373):GenerateWebPin : poid: **900105270**, poidLen: 9
DEBUG 08/22/16 23:15:53  pepper1be-d1-ap(2680/0)  junk.c(1408):GenerateWebPin : pinStr 
DEBUG 08/24/16 02:07:56  salt1be-d1-ap(5896/0)  uaadapter_vasco_totp.c(275):UAVascoTOTPImpl.close() -- Releasing Adapter Context
DEBUG 08/22/16 23:15:53  pepper1be-d1-ap(2680/0)  vsenterprise.cpp(288):VSEnterprise::Engage returns 0x4002 - Unknown error code **(0x4002)**
ERROR 08/22/16 23:15:53  pepper1be-d1-ap(2680/0)  vsautoauth.cpp(696):OSAAEndUserEnroll: error occurred. **Roll back** all updates!
INFO 08/24/16 02:07:56  salt1be-d1-ap(5896/0)  uaotptokenstoreqmimpl.cpp(199):Close token store
INFO 08/24/16 02:07:56  salt1be-d1-ap(5896/0)  main.c(990):-- EndOperation
-- 08/24 02:07:56 [mds.ecas(5896) ECAS_CP1] **==== END   ====**
    OPERATION = AutoAuthOSUserSubmit, rc = 0x0 (0)
    SYSINFO Elapse = 0.687, Heap = 1334K, Stack = 64K
</code></pre>

<p>It looks like you are simply trying to find strings within the LOG document and trying to parse the lines of characters using keywords. You can go line by line which is what you are doing currently or you could go through the document once (assuming the LOG document never gets huge) and add each subsequent line to an existing string.</p>

<p>Check this out for finding substrings
<a href="http://www.tutorialspoint.com/python/string_index.htm" rel="nofollow">http://www.tutorialspoint.com/python/string_index.htm</a> &lt;--- for finding the location of where a string is within another string, this will help you determine a start index and an end index. Once you have those you can extract your desired information.</p>

<p>Check this out for your CSV problem
<a href="http://www.tutorialspoint.com/python/string_split.htm" rel="nofollow">http://www.tutorialspoint.com/python/string_split.htm</a> &lt;--- for splitting a string around a specific character i.e. "," for your CSV files.</p>

<p><a href="http://stackoverflow.com/questions/3437059/does-python-have-a-string-contains-substring-method">Does Python have a string contains substring method?</a> will be more useful than your current method of using the strip() method</p>

<p>Hopefully this will point you in the right direction!</p>

python