Let’s throw some “Torch” on Tensor Operations

This article was published as a part of the Data Science Blogathon.

Discussing 5 Basic and Most Used Tensor Operations

Deep learning allows us to carry out a very wide range of complicated tasks. In order to carry out our tasks effectively, we need a tool that is flexible. Pytorch gives us this option because of its simplicity. It provides accelerated operations using GPUs (Graphical Processing Units). Due to this reason of Pytorch being a high-performance library that it has gained its popularity. The below notebook consists of some essential functions which are very useful in carrying out tensor operations. These operations are used for multi-dimensional tensors & for arithmetic operations.

• General Ops — Inverse
• Creation Ops — Complex
• Arithmetic Ops — Transpose
• Mutating Ops — Add
• Reduction Ops — Amax

We will discuss the examples of these 5 Basic functions & observe the errors. Before we begin, let’s install and import PyTorch

# Windows
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

# Import torch and other required modules
import torch

1. General Operations — Inverse Function

The first function we will be using is the ‘Inverse’ function.

a = torch.randn(2,3,3)
print(a)
torch.inverse(a)

The above ‘randn’ function has created a 3X3 square matrix with 2 outermost rows. The ‘inverse’ function then takes the inverse of the individual elements of the matrix

a = torch.rand(4,4)
print(a)
torch.inverse(a)

The above is a 4X4 square matrix where each element is inversed using the inverse function.

a = torch.rand(4,3)
print(a)
torch.inverse(a)

RuntimeError: A must be batches of square matrices, but they are 3 by 4 matrices

The third example given above throws an error. ‘Inverse’ function gives the inverse of the individual elements of the matrix. The error is due to the fact that the matrix is not square. By changing the dimensions, one can obtain the correct result.

The inverse function is very useful for performing an inverse function on the Pytorch neural network.

2. Creation Operations — Complex Function

Let’s use functions for initializing working on tensor data by creating a vector or matrix. Here we will be using the complex functions.

In order to get the resultant complex64, we need to input the float32 type.

real = torch.tensor([2,1], dtype=torch.float32)
imag = torch.tensor([2,3], dtype=torch.float32)
a= torch.complex(real, imag)
a.dtype

Here below, we have created ‘real’ & ‘imag’ named tensors using the rand function. Using the ‘complex’ function we have joined the two tensors & formed a single equation having real & imaginary numbers

real = torch.rand(2,3)
imag = torch.rand(1,3)
print(real)
print(imag)
x = torch.complex(real, imag,)
print(x)

Here, in the below example, instead of creating a complex tensor with two values ‘real’ & ‘imag’ data, it is trying to create a single complex tensor. We could see the above error just because of missing a single square bracket that would have given us the required result.

real = torch.tensor(2., 4)
imag = torch.tensor(7., 3)
x = torch.complex(real, imag,)
x

TypeError: tensor() takes 1 positional argument but 2 were given

We can use the above function for creating complex tensors consisting of real & imaginary data.

3. Mutating Tensor Data — Transpose Function

Here we will use the transpose function to work on mutating tensor data for making our operations easy.

a = torch.rand(2,3,5)
print(a)
torch.transpose(a,1,2)

From the outermost row1, we have transposed all the elements of the first row.

a = torch.rand(2,5)
print(a)
torch.transpose(a, -1, 0)

Here, in the above case, we are giving the first dimension & the second dimension to be transposed.

a = torch.rand(2,5)
print(a)
torch.transpose(a)

TypeError: transpose() received an invalid combination of arguments - got (Tensor), but expected one of:  * (Tensor input, name dim0, name dim1)  * (Tensor input, int dim0, int dim1)

While using the transpose function on the tensor data, we also have to pass the dimensions in order to clarify which dimensions need to be transposed. The above function would have worked accurately if we have used the ‘t’ instead of the ‘transpose’ function.

We can use the ‘transpose’ function when we have to transpose the given dimensions of tensor data while specifying which ‘n’ dimensions need to be transposed.

Arithmetic Operations — Add Function

Let’s perform some arithmetic operations — add a function on our tensor data.

a = torch.randn(10)
print(a)
torch.add(a,5)

The second attribute (5 in the above case) should be an integer that must be added to the tensor data (as in the above case). The resultant will be the sum of two.

a = torch.rand(5)
b = torch.rand(5)
print(a)
print(b)
torch.add(a,b)

‘Add’ function computes the sum of two tensor data of the same dimensions & gives the result in the same dimension.

a = torch.rand(10)
b = torch.rand(5)
torch.add(a,b)

RuntimeError: The size of tensor a (10) must match the size of tensor b (5) at non-singleton dimension 0

While performing any arithmetic operations in tensor, we need to take care that the dimensions of the input tensors match each other.

‘Add’ function can be used to add any two given tensors, or also to add tensor data with a given number.

Reduction Operations — Amax Function

Using certain Reduction operations — amax. These will help in performing statistical operations on our tensor data. Here, in the below example, ‘amax’ function is used to give the maximum element in each dimension, where ‘-1’ shows the dimension to be reduced.

a = torch.rand(3,2)
print(a)
torch.amax(a, dim = -1)

Also, in the below case, ‘amax’ function gives us the maximum value in the tensor data each for each slice.

a = torch.rand(5)
print(a)
torch.amax(a, dim=-2)

In the below case, the dimensions of amax function vary from -1 to 0. Hence, ‘dim’ attribute must be within this range.

a = torch.tensor([[3,2], [1,2], [4,7],[6,5]])
print(a)
torch.amax(a, dim = 1)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got -2)

Conclusion

In this topic, we have covered the necessary functions from the creation of tensor data to carrying out arithmetic operations. We have also carried out mutating operations which are very much necessary for the general understanding.

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.

Anchit