Linear Regression in TensorFlow is easy to implement.

Linear regression fits a straight line through your data. The line has a slope and an intercept, and the model learns both by looking at the error it makes.

In TensorFlow you write that line as a formula with placeholders for x and y, then train it with the train operation.

In the Linear Regression Model:
The goal is to find a relationship between a scalar dependent variable y and independent variables X.

The model is based on real world data and can be used to make predictions. Of course you can use random data, but it makes more sense to use real world data.

The model

Consider this example:

X: gdp per capita
Y: life expectancy
Predict Y from X

The model is:

Y_predicted = X * w + b

Linear regression with tensorflow

You can create a linear regression prediction model in a few steps.
If you want you can see the graph with TensorBoard.

You can find the complete code and dataset in this repo.

  1. read the data
    This can be a simple text file with tab separated values.

  2. create placeholders for inputs

    X = tf.placeholder(tf.float32, name='X')
    Y = tf.placeholder(tf.float32, name='Y')

  3. create weights and bias

    w = tf.get_variable('weights', initializer=tf.constant(0.0))
    b = tf.get_variable('bias', initializer=tf.constant(0.0))

  4. make model to predict

    Y_predicted = w * X + b

  5. define loss function

    # can use square error as loss function
    loss = tf.square(Y - Y_predicted, name='loss')

  6. create optimizer

    optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.0003).minimize(loss)

  7. train model (initialize variables, run optimizer)

  8. plot (optional)

The steps

  • Read the data into two numpy arrays, x and y.
  • Create placeholders for them so TensorFlow can feed data later.
  • Define the model, for one feature it is y = w * x + b.
  • Use tf.reduce_mean(tf.square(y - y_pred)) as the loss, that is the mean squared error.
  • Call tf.train.GradientDescentOptimizer and run it with sess.run(train) in a loop.
  • Print the learned w and b after training, then use them to predict.

Always print the loss every so often. If it goes down the learning rate is fine, if it goes up you should use a smaller one.

If you want to try this yourself, the exercises on PyChallenge turn the same idea into a few short problems with instant feedback.