# Yekun's Note

Machine learning notes and writeup.

Here is some useful notes/tricks of Tensorflow 1.x, a powerful deep learning framework developed by Google, Inc.

# Op

## tensor

1. eval() [2]

2. tf.group()

3. get the batch size from placeholder

# Tensor manupulation

## tf.gather

• tf.gather slices the params with indix indices along axis.
• output shape = params.shape[:axis] + indices.shape[batch_dims:] + params.shape[axis+1:]. The middle term indicates the axis to slice on.

## tf.gather_nd

• Slice the params with the specified shape of indices.
• Slice on the first N dims, where N=indices.shape[-1]. i.e., # of the slice op.
• indices.shape[-1] <= params.rank
• if equal, slice the element
• if not equal, slice along the indices.shape[-1] axis.
• out.shape = indices.shape[:-1] + params.shape[indices.shape[-1]:]. indices.shape[-1] indicate the dim after slicing.

1. tf.max_band_part
Copy a tensor setting everything outside a central band in each innermost matrix to zero. That is, elements below the “num_lower” (if not -1) and above the “num_upper” are set to zeros.

## tf.clip_by_averge_norm

where $m$ is the number of tensor $t$.

# tf.distributions

## Multinormial

Sample from probs and get the probabilities

# Categorical

Intuited as generating sample of argmax{ OneHotCategorical(probs)} itself being identical to argmax{ Multinomial(probs, total_count=1)}.

# Control flow

image source: [5]

## tf.cond

• Forward pass tf.cond(pred, fn1, fn2)
• Gradient tf.cond(pred, grad(fn1), grad(fn2))

## tf.while_loop

• Forward pass tf.while_loop(cond_fn, body_fn, loop_vars) -> executes N times

# Eager execution mode

Eager execution mode support the dynamic graph and print out the values of tensors when creating graphs without using tf.Session() for version 1.x. [4]

## 12# start the file withtf.enable_eager_execution()

• Get undefined Placeholder op names

# Configuration

## GPU designation

• Designate GPU in tf[7][8]

• Check GPU usage

• Peridically watch the gpu usage:

## Log

Level Level for Humans Level Description
0 DEBUG [Default] Print all messages
1 INFO Filter out INFO messages
2 WARNING Filter out INFO & WARNING messages
3 ERROR Filter out all messages