spark dataframe read text file without headers
<p>i have a text file with no headers, how can i read it using spark dataframe api and specify headers. Is there a way to specify my schema</p>

<p>sample_data = spark.read.option("header", "false").text(sample)</p>

<p>print "Data size is {}".format(sample_data.count())</p>

<p>print type(sample_data)</p>

<p>print sample_data.take(2)</p>

<p>First, save your file as csv. You can specify the schema:</p>

<pre><code>schema = StructType([ \
    StructField("column1", StringType(), True), \
    StructField("column2", DoubleType(), True), \
    StructField("column3", IntegerType(), True)])
</code></pre>

<p>And so on.
If you're using spark 2.0 +:</p>

<pre><code>spark.read.csv(
    "file.csv", header=True, schema=schema
)
</code></pre>

<p>If you're using spark &lt; 2.0:</p>

<pre><code>sales = sqlContext.read.format('com.databricks.spark.csv')\
    .options(header='true', delimiter='whatever youre using as delimiter')\
    .load('file.csv', schema = schema)
</code></pre>

python apache-spark dataframe pyspark