Tensorflow: Filter must not be larger than the input
<p>I want to perform convolution along the training sample that is of shape  [n*1] and apply zero-padding too. So far, no results.</p>

<p>I am building a character-level CNN (idea taken from <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/skflow/text_classification_character_cnn.py" rel="nofollow">here</a>)</p>

<p>My data is essentially tweets, initially each of length 140. 
I filter all non-alphabetical characters (replace them with empty string ''), all alphabetical characters I convert to lowercase and encode as one-hot encoder. </p>

<p>So I have data is n*m, where n is the number of training examples, m=140*26=3640, since each alphabetical character is encoded as one-hot vector.</p>

<p>Now, I am trying to perform convolution, and here is where I have a problem. Essentially:
1) I try to pad a single tweet with zeros around it. 
2) Then what I want to do is perform convolution with a filter 3*3 along the tweet, which, I expect to be of 3642*3 size, where width = 3642 and height=3 after padding.</p>

<pre><code>F = 3 # filter size
S = 1 # stride
P = 1 # zero-pading
MAX_DOCUMENT_LENGTH = 3640
IMAGE_WIDTH = MAX_DOCUMENT_LENGTH
IMAGE_HEIGHT = 1
N_FILTERS = 20
FILTER_SHAPE1 = F
BATCH_SIZE = 257

def conv_model(X, y):
    X = tf.cast(X, tf.float32)
    y = tf.cast(y, tf.float32)
    # reshape X to 4d tensor with 2nd and 3rd dimensions being image width and height
    # final dimension being the number of color channels
    X = tf.reshape(X, [-1, IMAGE_WIDTH, IMAGE_HEIGHT, 1])
   # first conv layer will compute N_FILTERS features for each FxF patch
    with tf.variable_scope('conv_layer1'):
            h_conv1 = tf.contrib.layers.conv2d(inputs=X,num_outputs=N_FILTERS, 
                                  kernel_size=[3,3], padding='VALID')
</code></pre>

<p>I get the error: <code>ValueError: Filter must not be larger than the input: Filter: (3, 3) Input: (3640, 1)</code></p>

<p><strong>Why is zero-padding not applied?</strong> At least, the result of its application does not work...</p>

<p>So I change the filter size to <code>[3,1]</code> and I call:</p>

<pre><code>h_conv1 = tf.contrib.layers.conv2d(inputs=X, num_outputs=N_FILTERS, kernel_size=[3,1], padding='VALID')
</code></pre>

<p>And I don't get the error. </p>

<p><strong>Could someone please explain what is happening?</strong> </p>

<p>Also, why do we need to reshape the input as X = tf.reshape(X, [-1, IMAGE_WIDTH, IMAGE_HEIGHT, 1])? </p>

<blockquote>
  <p>Why is zero-padding not applied?</p>
</blockquote>

<p>Use <code>padding = 'SAME'</code> in conv2d for zero-padding.</p>

<blockquote>
  <p>Could someone please explain what is happening?</p>
</blockquote>

<p>You can't use the 3x3 filter in the case of 'flat' image. To use the 3x3 filter, input should have dimensions bigger than 3 on both width and height. </p>

<blockquote>
  <p>Also, why do we need to reshape the input as X = tf.reshape(X, [-1,
  IMAGE_WIDTH, IMAGE_HEIGHT, 1])?</p>
</blockquote>

<p>A Single image has a shape of [width, height, number_of_channels]. Extra dimension stands for minibatch size. -1 just preserves total size.</p>

python machine-learning tensorflow deep-learning convolution