Top 3 Ways to Write Your Tensorflow Code

This article was published as a part of the Data Science Blogathon

Having choices gives us flexibility and scope of creativity

Introduction

I know that not all the choices available to us in our life would make us feel good. But having them makes us feel better and less trapped even if these choices are not the best ones. But today it’s not a matter of survival choices just some productivity choices, these 3 choices are a few out of many others out there.

In this article, we will look at 3 ways in which we can use TensorFlow and Keras to create deep learning models AKA Neural Networks. To get all of us on the same page, let’s start with an introduction.

Tensorflow: It is an end-to-end open-source platform managed and developed by Google for machine learning.

Keras: It is also an open-source software library that provides a Python interface for deep learning neural networks. Keras acts as an interface for the Tensorflow Library.

Before moving along with the content, I want to make this clear that for one of the ways, you need to have good knowledge about object-oriented programming (Classes and Objects). If in case you are not familiar with the concepts, no problem just looks at the other 2 for teaching yourself those methods.

Now, as mentioned above, Keras acts as a high-level API through which we will interact with most of the TensorFlow code. All these methods are used through Keras API.

1. The Sequential Model API

Just like any sequence in real-world or in programming terms, a sequence is an arrangement of elements in some particular order. With sequential API, we easily get to make a plain stack of layers where each layer has exactly one input tensor and one output tensor.

Image Link

Image Link

Understand this better with this example:

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# Define Sequential model with 3 layers
model = keras.Sequential(
    [
        layers.Dense(2, activation="relu", name="layer1"),
        layers.Dense(3, activation="relu", name="layer2"),
        layers.Dense(4, name="layer3"),
    ]
)
# Call model on a test input
x = tf.ones((3, 3))
y = model(x)

In the above code, you have stacked up 3 Dense layers on top of each other to create a single pathway through them. This will be equivalent to writing this:

# Create 3 layers
layer1 = layers.Dense(2, activation="relu", name="layer1")
layer2 = layers.Dense(3, activation="relu", name="layer2")
layer3 = layers.Dense(4, name="layer3")

# Call layers on a test input
x = tf.ones((3, 3))
y = layer3(layer2(layer1(x)))

Where the output ‘y’ is produced after the input x goes from one layer to another in a single fixed sequence.

One important thing to keep in mind is that all these methods have their strengths and weaknesses. Let us glance over when Sequential is not so appropriate to use:

1. Multiple Inputs and Multiple Outputs: When your network architecture is designed for a task that requires input from more than one input layer or let’s say different inputs are being used parallelly for the execution, using a sequential layer is not the best approach. The same goes for multiple outputs from the network.

2. Any of the individual layers has multiple inputs or multiple outputs: Now it is similar to the above point but not exactly the same. There are various elements involved behind a neural network and let’s say that your architecture needs one or more of the hidden layers to perform certain tasks that are coming from the outside like a ‘T’ joint, then that would not be possible with sequential stacking of layer, they are fixed for a single path without any divergence or junctions.

3. You need to do layer sharing: Layer sharing is done when you want to use the output or let’s say behaviour of one layer and use it in accordance with another layer. That kind of sharing is not possible with sequential modelling through this API.

4. You want non-linear topology: If you are creating something like residual connections, or a multi-branch model.

2. The Functional API

The Keras functional API is a great way to create models that are more flexible than the Sequential Model API. The Functional API can easily handle models with non-linear topology, shared layers, and even multiple inputs or outputs. The main idea behind it that is quite clear to understand is that a deep learning model is usually a directed acyclic graph of layers. So the functional AP is a way to build graphs of layers.

Now, instead of stacking them all together and had no involvement in handling the hidden layer’s inputs and outputs, with functional API way, we get to use the layers as functions. So, the parameter of the function is input and what they produce will be the output to use further.

Image Link

Let’s see it in action.

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

we will create a basic graph with three layers like above. To start we create an Input Node.

inputs = keras.Input(shape=(784,))

The shape specified as a parameter is the shape of the input data which is set to a 784-dimensional vector. The batch size is always omitted since only the shape of each sample is specified.

Further making the other layer (Dense) who will use ‘inputs’ object as their input and so this cascading effect follows.

dense = layers.Dense(64, activation="relu")
x = dense(inputs)

x = layers.Dense(64, activation="relu")(x)
outputs = layers.Dense(10)(x)

Closely observe the cascading of layers through function calls. At last import input layer and output layer as the argument for making an object of Model call, which will define our neural network.

model = keras.Model(inputs=inputs, outputs=outputs, name="mnist_model")

As an experimental task for you, after completing the Model object definition. Call .summary() on the ‘model’ object.

model.summary()

3. The Subclassing API

This approach required you to understand the concepts of object-oriented programming (Classes and Objects). If you don’t know about them, it’s completely okay to not know it, just skip this method and focus on the above two.

For those who are familiar with the concepts, let’s get started!

While we are using the method, there are various ways we can inherit other parent classes and use their functionality to even create our own layers and model. Today, I will talk about making your own model with subclassing API. So the first step you do is to define a class that contains all the code for your model architecture and the calling of those layers in whichever order you decide (very versatile and flexible).

Image Link

We will use the concept of inheritance and use the functionalities and methods available to use in tensorflow.keras.Model Class. One more thing before you jump on to the code, DON’T FORGET TO CLASS THE SUPER() FUNCTION, which will direct the access to the parent class and help to call the constructor function of the parent class (tensorflow.keras.Model).

class MyModel(tf.keras.Model):

    def __init__(self, num_classes=2):
        super(MyModel, self).__init__()
        self.dense1 = tf.keras.layers.Dense(64, activation="relu")
        self.dense2 = tf.keras.layers.Dense(64, activation="relu")
        self.classifier = tf.keras.layers.Dense(num_classes)

    def call(self, inputs):
        x = self.dense1(inputs)
        x = self.dense2(x)
        return self.classifier(x)


model = MyModel()
dataset = ...
model.fit(dataset, epochs=10)
model.save(filepath)

__init__(): this is the constructor which will execute as soon as we define an object from this class.

call(): this function is used when we are calling the object that we just created from the class for the purpose of transformation from inputs to outputs.

Then things are pretty much the same that they were when we create a model and fit on it with some data.

Visual Comparison of these API structures (Again!):

Image Link

I hope you found this article useful and got some ideas about how differently you can create your models using these various methodologies.

Gargeya Sharma

B.Tech 3rd Year Student
Specialized in Deep Learning and Data Science
For getting more info check out my Github Homepage

LinkedIn GitHub

Gargeya