How do I add headers for the output csv for apache beam dataflow?
<p>I noticed in the java sdk, there is a function that allows you to write the headers of a csv file. 
<a href="https://cloud.google.com/dataflow/java-sdk/JavaDoc/com/google/cloud/dataflow/sdk/io/TextIO.Write.html#withHeader-java.lang.String-" rel="nofollow">https://cloud.google.com/dataflow/java-sdk/JavaDoc/com/google/cloud/dataflow/sdk/io/TextIO.Write.html#withHeader-java.lang.String-</a></p>

<p>Is this features mirrored on the python skd? </p>

<p>This is not implemented at this moment. However you can implement/extend it yourself (see <a href="https://gist.github.com/Fematich/97703910d867f972e9d01b21d8f41221" rel="nofollow">attached notebook</a> for an example+test with my version of apache_beam).</p>

<p>This is based on a <a href="https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/io/fileio.py" rel="nofollow">note in the docstring</a> of the superclass <code>FileSink</code>, mentioning that you should overwrite the <code>open</code> function:</p>

<p>The new class that works for my version of apache_beam ('0.3.0-incubating.dev'):</p>

<pre><code>import apache_beam as beam
from apache_beam.io import TextFileSink
from apache_beam.io.fileio import ChannelFactory,CompressionTypes
from apache_beam import coders


class TextFileSinkWithHeader(TextFileSink):
    def __init__(self,
               file_path_prefix,
               file_name_suffix='',
               append_trailing_newlines=True,
               num_shards=0,
               shard_name_template=None,
               coder=coders.ToStringCoder(),
               compression_type=CompressionTypes.NO_COMPRESSION,
               header=None):
        super(TextFileSinkWithHeader, self).__init__(
            file_path_prefix,
            file_name_suffix=file_name_suffix,
            num_shards=num_shards,
            shard_name_template=shard_name_template,
            coder=coder,

            compression_type=compression_type,
            append_trailing_newlines=append_trailing_newlines)
        self.header = header

    def open(self, temp_path):
        channel_factory = ChannelFactory.open(
            temp_path,
            'wb',
            mime_type=self.mime_type)
        channel_factory.write(self.header+"\n")
        return channel_factory
</code></pre>

<p>You can subsequently use it as follows:</p>

<pre><code>beam.io.Write(TextFileSinkWithHeader('./names_w_headers',header="names"))
</code></pre>

<p>See <a href="https://gist.github.com/Fematich/97703910d867f972e9d01b21d8f41221" rel="nofollow">the notebook</a> for the complete overview.</p>

python google-cloud-dataflow apache-beam