How to import CSV file to django models
<p>I have Django models like this</p>

<pre><code>class Revo(models.Model):           
    SuiteName = models.CharField(max_length=255)
    Test_Case = models.CharField(max_length=255)
    FileName = models.CharField(max_length=255)
    Total_Action = models.CharField(max_length=255)
    Pass = models.CharField(max_length=255)
    Fail = models.CharField(max_length=255)
    Exe_Time = models.CharField(max_length=255)
    Result = models.CharField(max_length=255)
    create_date = models.DateTimeField(default=datetime.datetime.now)

    class Meta:
        verbose_name_plural = "Revo"
</code></pre>

<p>I have CSV file like this</p>

<pre><code>SuiteName,Test Case,FileName,Total Action,Pass,Fail,Exe Time,Result
DEMO_TEST_SUITE,Testcase 1,file1,82,0,108,0:27:52,FAIL
DEMO_TEST_SUITE,Testcase 2,file2,86,0,108,0:27:52,FAIL
DEMO_TEST_SUITE,Testcase 3,file3,820,0,108,0:27:52,FAIL
DEMO_TEST_SUITE,Testcase 4,file4,182,0,108,0:27:52,FAIL
DEMO_TEST_SUITE,Testcase 5,file5,102,0,108,0:27:52,FAIL
DEMO_TEST_SUITE,Testcase 6,file6,111,0,108,0:27:52,FAIL
</code></pre>

<p>How do I import this csv data into my django models? Also, is there any way to plot graphs out this data directly from database?</p>

<p><strong>Before you start</strong></p>

<p>The first thing to do is to normalize your database. The data in the CSV file isn't normalized - it need not be. However there is no reason for your table to reflect the structure of your CSV file.</p>

<p>For exaple <code>DEMO_TEST_SUITE</code> is repeated and there's no reason to store 'Testcase 1', 'Testcase 2' etc. It's far more efficient to store just the 1,2,.. in that column.</p>

<pre><code>class Revo(models.Model):           
    SuiteName = models.ForeignKey(Suite)
    Test_Case = models.IntegerField()
    FileName = models.CharField(max_length=255)
    Total_Action = models.IntegerField()
    Pass = models.CharField(max_length=255)
    Fail = models.CharField(max_length=255)
    Exe_Time = models.CharField(max_length=255)
    Result = models.IntegerField()
    create_date = models.DateTimeField(default=datetime.datetime.now)

    class Meta:
        verbose_name_plural = "Revo"
</code></pre>

<p>Some of the other fields should really be integers as well instead of varchar(255). But before you the changes, make a table with the same structure as your current one, it's going to be usefull too!</p>

<p><strong>Doing the import.</strong></p>

<p>Creating a CSV reader, reading each line and creating records one by one is probably the last thing you want to do. This will be slow. Very slow if you have even a hundred thousand records.</p>

<p>What you really ought to do is to use <a class='doc-link' href="http://stackoverflow.com/documentation/mysql/2356/load-data-infile#t=201610112255043619058">LOAD DATA INFILE</a> and if you are using postgresql, you ought to be using the <a href="https://www.postgresql.org/docs/9.5/static/sql-copy.html" rel="nofollow">COPY</a> and in the case of Sqlite, you ought to use the <a href="https://www.sqlite.org/cvstrac/wiki?p=ImportingFiles" rel="nofollow">.import</a> command. These are built in mechanisms for import CSV data that operate very fast and they have the capability to handle duplicates as well.</p>

<p>Import into the 'old' table, create the foreign keys and the SELECT INTO the proper table represented by your normalized model.</p>

python django csv graph