In TensorFlow you do not write the program straight away. You first build a graph: a description of the operations and the order they run in. Then you open a session and run the graph with data.

That split is what lets you look at the graph in TensorBoard before anything runs.

That split is what lets you look at the graph in TensorBoard before anything runs.

You can create large graphs and graphs with subgraphs. You can visualize this with TensorBoard.

Graph

In tensorflow we define graphs. These graphs show the operations in the session. The example below creates a graph with more nodes:

import tensorflow as tf

x = 1
y = 6

op1 = tf.add(x,y)
op2 = tf.add(x,y)
op3 = pow(op1, op2)

with tf.Session() as ses:
    writer = tf.summary.FileWriter('./graphs', ses.graph)
    print(ses.run(op3))

Save it and run it with Python. Then open it with TensorBoard and this graph should show up:

tensorflow graph

Subgraphs

Subgraphs are also possible, you could have something like:

op1 = tf.add(x,y)
op2 = tf.add(x,y)
op3 = tf.pow(op1, op2)
thing = tf.multiply(x, op1)

A subgraph is possible to break into chunks, each can be run on a different CPU or GPU.

Can you make more than one graph?

You can build more than one graph, but that doesn’t work distributed.

The session runs the default graph. You should have disconnected subgraphs in a graph.

Graphs and sessions

  • Nodes are operations, edges are the data that flows between them.
  • tf.Graph() makes a graph, as_default() makes it the one you build into.
  • tf.Session() runs the graph.
  • A session owns the variables and the memory, so close it when you are done.
  • With tf.compat.v1 you still need both, TensorFlow 2 hides this.

Subgraphs let you break a big model into chunks and run parts of them on different CPUs or GPUs. Write the file while building the graph, then open it in TensorBoard.

If you want to try this yourself, the exercises on PyChallenge turn the same idea into a few short problems with instant feedback.