Menu

TensorFlow Code Snippets

TensorFlow

Quick Reference for TensorFlow

Basics

Playground:  http://playground.tensorflow.org/
Activation Functions:  https://ml-cheatsheet.readthedocs.io/en/latest/activation_functions.html#softmax
CNN:  https://poloclub.github.io/cnn-explainer/
TF Hub:  https://www.tensorflow.org/hub
Papers:  paperswithcode.com
Weights: www.wandb.ai

Overcome assertion error by installing (in order):

  • CUDA: https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exe_local  (download exe to machine)
  • PyTorch: https://pytorch.org/get-started/locally/  (should look similar to: !conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge) and yes, it takes a while.
  • Test at cmd prompt:  import torch; x = torch.rand(5,5), print (x) –> a 5×5 vector of numbers should display;
  • Test torch at cmd prompt:  import torch; torch.cuda.is_available() –>

# Create random tensors
random_1 = tf.random.Generator.from_seed(42) #set seed for reproducibility   #global level
random_1 = random_1.normal(shape=(3,2))  #operational level

# Create tensor from NumPy array
import numpy as np
numpy_A = np.arange(1, 25, dtype = np.int32)

# Change into a tensor: convert NumPy array into a tensor
A = tf.constant(numpy_A, shape = (2,3,4))  #needs to have the same number of elements
B = tf.constant(numpy_A)
C = tf.constant(numpy_A, shape = (3,8)) #equals 24

# Squeeze a tensor (removes all dimensions of size 1)
tensor_squeezed = tf.squeeze(tensor_to_squeeze)

tf.constant(y_preds_A), tf.squeeze(y_preds_A) #need to add constant to see the shape

# Change from int32 to float32
# Reduce precision: default is 32 but float-16 only uses 16-bits of memory instead of 32-bits so it runs faster
C = tf.constant([13, 29])
E = tf.cast(C, dtype = tf.float32)
E2 = tf.cast(C, dtype = tf.float16)

# Add another dimension to the tensor
rank_5_tensor = rank_4_tensor[…, tf.NewAxis]   #… means add new axis to the end

# View tensor attributes
rank_4_tensor = tf.zeros(shape=[2,3,4,5])
print (“Datatype of every element:”, rank_4_tensor.dtype)
print(“Number of dimensions (rank):”, rank_4_tensor.ndim)
print (“Shape of tensor:”, rank_4_tensor.shape)
print (“Element along the 0 axis:”, rank_4_tensor.shape[0])
print (“Element along the last axis:”, rank_4_tensor.shape[-1])
print (“Total number of elements in our tensor:”, tf.size(rank_4_tensor))
print (“Total number of elements in our tensor:”, tf.size(rank_4_tensor).numpy()) #visually cleaner

# Get the first element from each dimension from each index except for the final one
rank_4_tensor[:1, :1, :1, :1]

# Matrix Multiplication aka Dot Product

tf.matmul(tensor_a, tensor_b)

# Matrix multiplication with Python operator “@”
tensor @ tensor

# Change the shape of a tensor
tf.reshape(Y, shape=(2, 3))

# Variance and Standard Deviation

std = tf.math.reduce_std(E, keepdims=False)
var = tf.math.reduce_variance(E)

# Find the positional maximum / minimum
tf.argmax(F)
tf.argmin(F)
# View the maximum value
F[tf.argmax(F)]
F[tf.argmin(F)]

# One-hot encode list of indices

some_list = [15, 25, 88, 99]
tf.one_hot(some_list, depth=4)
tf.one_hot(some_list2, depth = 4, on_value = “Onnnn”, off_value = “offfff”)

# If there is a function that doesn’t work with the data types being used, can move it to a NumPy array and then back. Be aware of the basic data types i.e. float 16 vs float32
# Convert tensor A to a NumPy array
tensorA.numpy(), type(tensorA.numpy())

# One-hot encoding dataframe with categorical features
pd.get_dummies(dataframe_name)

# Load Data from TensorFlow Datasets

import tensorflow_datasets as tfds
(train_data, test_data), ds_info = tfds.load(name = “file_name i.e. food101”,
split = [“train”, “validation”],
shuffle_files = True,
as_supervised = True,    # data gets returned in tuple format (data, label),
with_info = True)    #download image, label and metadata which is ds_info

Visualization

#layers
from tensorflow.keras.utils import plot_model
plot_model(model=model, show_shapes = True)

Overview of model:
model.summary()

# Plotting function

def plot_predictions(train_data = X_train,
train_labels = y_train,
test_data = X_test,
test_labels = y_test,
predictions = y_pred):
#plots training data, test data and compare predictions to ground truth
plt.figure(figsize=(10, 7))
plt.scatter(train_data, train_labels, c=”b”, label = “Training data”)
plt.scatter(test_data, test_labels, c=”g”, label = “Testing data”)
plt.scatter(test_data, predictions, c=”r”, label = “Predictions”)  #plot models predictions in red
plt.legend();

Run:  plot_predictions (train_data = X_train, train_labels = y_train, test_data = X_test, test_labels = y_test, predictions=y_pred)

#Load data

import tensorflow_datasets as tfds
datasets_list = tfds.list_builders()
(train_data, test_data), ds_info = tfds.load(name=”food101″, # dataset to get from TFDS
                                             split=[“train”, “validation”],  # what splits of the data to retrieve (not all datasets have train, valid, test so adj)
                                             shuffle_files=True,   # shuffle files on download
                                             as_supervised=True, # download data in tuple format (data sample, data label)
                                             with_info=True)  #include metadata so tfds returns a tuple of data, ds_info

#Dig-In to the Data

#Features of the data
ds_info.features

#Get class names
class_names = ds_info.features[“label”].names
class_names[:20]

#visualize the data
train_sample = train_data.take(1)  #select a sample
train_sample  #(what does the tensor look like)

# Output info about our training sample – remember its random so it will produce different results each time
for image, label in train_sample:
print(f”””
Image shape: {image.shape}
Image dtype: {image.dtype}
Target class from Food101 (tensor form): {label}
Class name (str form): {class_names[label.numpy()]}
“””)

# View the tensor
image

#View the min & max values (used later in normalization)
tf.reduce_min(image), tf.reduce_max(image)

#Plot one sample tensor with the label
import matplotlib.pyplot as plt
plt.imshow(image)
plt.title(class_names[label.numpy()])  #add a title to image by indexing on class_names list otherwise will return a number ref class
plt.axis(False);

Architecture

For Regression

For Classification

 

GPU Setup

# Note, this is note a two-minute job…long, tedious, and requires numerous reboots.  Plan appropriately 🙁

# Determine if GPU is connected:  tf.config.list_physical_devices()
# If not and you need to set it up then:
# Check Nvidia: https://www.geforce.com/geforce-experience/download #generally requires a reboot so be aware
# Ensure Visual Studio 2019 is installed: https://visualstudio.microsoft.com/vs/
# Ensure Desktop Development with C++ is installed along with: a) Windows 10 SDK – any version will work and b) MSVC v142 — VS 2019 C++ x64/x86 build tools (v14.24)
# Install the CUDA Toolkit *ensure its version 10*: https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal
# Run the appropriate version for Visual Studio should be at C:\ProgramData\NVIDIA Corporation\CUDA\Samples\v10.1\1_Utilities\deviceQuery
# Laptop not needed as CUDA already installed and available to all cores #device manager
# Per TF: https://www.tensorflow.org/install/gpu
# To upgrade: pip3 install –upgrade tensorflow-gpu
# NVIDA instructions: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#install-windows
# Ref: https://medium.com/analytics-vidhya/tensorflow-gpu-installation-with-cuda-cudnn-40fbd4477e7
# Go through all the steps #painful DNN module steps: https://medium.com/@amar.bit.jsr/dnn-module-build-with-cuda-support-on-windows-10-3a2f3a736ca