PyTorch_Tutorials_Tensor 공식사이트 튜토리얼

https://pytorch.org/tutorials/beginner/basics/tensorqs_tutorial.html

Tensors — PyTorch Tutorials 1.13.1+cu117 documentation

TENSOR

Tensors are a specialized data structure that are very similar to arrays and matrices. In PyTorch, we use tensors to encode the inputs and ouputs of a model, as well as the model's parameters.

 

Tensors are simliar to NumPy's ndarrays, except that tensors can run on GPUs or other hardware accelerators. In fact, tensros and NumPy arrays can often share the same underlying memory, eliminating the need to copy data. Tensors are also optimized for automatic differentiation. 

텐서는 Numpy의 ndarray와 유사하고 실제로 내부 메모리를 공유하고 있어 데이터를 복사할 필요가없다.

import torch
import numpy as np

Initializeing a Tensor

Tensors can be initialized in various ways.

 

Directly from data

Tensors can be created directly from data. The data type is automatically inferred.

data = [[1,2],[3,4]]
x_data = torch.tensor(data)

From a NumPy array

Tensors can be created from Numpy arrays. and vice versa

np_array = np.array(data)
x_np = torch.from_numpy(np_array)

From another tensor:

The new tensor retains the properties(shape, datatypes) of the argument tensor, unless explicitly overridden.

재정의(override)하지 않으면 속성과 자료형을 유지한다.

x_ones = torch.ones_like(x_data) # retains the properties of x_data
print(f"Ones Tensor: \n {x_ones} \n")

x_rand = torch.and_like(x_data, dtype=torch.float) # overrides the datatype of x_data
print((f"Random Tensor: \n {x_rand}\n")

# Out print
Ones Tensor:
 tensor([[1, 1],
        [1, 1]])

Random Tensor:
 tensor([[0.4386, 0.6174],
        [0.7119, 0.4048]])

With random or constant values:

shape is a tuple of tensor dimensions. In the functions below, it determines the dimensionality of the output tensor.

shape은 차원을 결정하는 텐서

shape = (2,3,)
rand_tensor = torch.rand(shape)
ones_tensor = torch.ones(shape)
zeros_tensor = torch.zeros(shape)

print(f"Random Tensor: \n {rand_tensor}\n")
print(f"Ones Tensor: \n {Ones_tensor}\n")
print(f"Zeros Tensor: \n {Zeros_tensor}\n")

# out print
Random Tensor:
 tensor([[0.5633, 0.9897, 0.6948],
        [0.9888, 0.4163, 0.8700]])

Ones Tensor:
 tensor([[1., 1., 1.],
        [1., 1., 1.]])

Zeros Tensor:
 tensor([[0., 0., 0.],
        [0., 0., 0.]])

Attribue of a Tensor

Tensor attributes describe their shape, datatype, and the device on which they are stored.

tensor = torch.rand(3,4)

print(f"Shape of tensor: {tensor.shape}")
print(f"Datatype of tensor: {tensor.dtype}")
print(f"Device of tensor: {tensor.device}")

# out print
Shape of tensor: torch.Size([3, 4])
Datatype of tensor: torch.float32
Device tensor is stored on: cpu

Operations on Tensors

Over 100 tensor operations, including arithmetic, linear algebra, matrix manipulation(transposing, indexing, slicing).

Each of these operations can be run on the GPU (at typically higher speeds than on a CPU).

By default, tensors are created on the CPU. We need to explicitly move tensors to the GPU using .to method (after checking for GPU availability). Keep in mind that copying large tensors across devices can be expensive in terms of time and memory!

텐서는 기본적으로 CPU에 생성되지만 .to 메서드로 GPU로 이동할 수 있다.(GPU가 연산이 더 빠르다.)

# We move our tensor to the GPU if available
if torch.cuda.is_available():
	tensor = tensor.to("cuda")

Standard numpy-like indexing and slicing:

tensor = torch.ones(4,4)
print(f"First row: {tensor[0]}")
print(f"First cloumn: {tensor[:,0]}")
print(f"Last clomun: {tensor[...:-1]}")
tensor[:,1]=0
print(tensor)

First row: tensor([1., 1., 1., 1.])
First column: tensor([1., 1., 1., 1.])
Last column: tensor([1., 1., 1., 1.])
tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])

Joing tensors

You can use torch.cat to concatenate a sequence of tensors along a given dimension.

t1 = torch.cat([tensor, tensor, tensor], dim=1)
print(t1)

tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])

Arithmetic operations

# This computes the matrix multiplication between two tensors.
# y1, y2, y3 will have the same value
# ''tensor.T'' returns the transpose of a tensor
y1 = tensor @ tensor.T
y2 = tensor.matmul(tensor.T)

y3 = torch.rand_like(y1)
torch.matmul(tensor, tensor.T, out=y3)

# This computes the element_wise product. z1, z2, z3 will have the same value
z1 = tensor * tensor
z2 = tensor.mul(tensor)

z3 = torch.rand_like(tensor)
torch.mul(tensor, tensor, out=z3)

tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])

Single-element tensors

If you have a one-element tensor, for example by aggregating all values of a tensor into one value, you can convert it ot a Python numerical value using item():

agg = tensor.sum()
agg_item = agg.item()
print(agg_item, type(agg_item))

12.0 <class 'float'>

In-place operations

Operations that store the result into the operand are called in-place.

They are denoted by a _ suffix. For exaple: x.copy_(y), x.t_(), will change x.

변수를 만들지 않고 값을 저장할 수 있다.

print(f"{tensor}\n")
tensor.add_5()
print(tensor)

tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])

tensor([[6., 5., 6., 6.],
        [6., 5., 6., 6.],
        [6., 5., 6., 6.],
        [6., 5., 6., 6.]])

※ 바꿔치기 연산은 메모리를 일부 절약하지만, 기록이 즉시 삭제되어 도함수계산에 문제가 발생할 수 있으므로 사용을 권장하지 않는다.

 

Bridge with Numpy

Tensors on the CPU and Numpy arrays can share their underlying memory locations, and chaning one will change the other.

메모리를 공부하므로 (CPU상에서) 하나를 변경하면 다른 하나도 변경됨

t = torch.ones(5)
print(f"t: {t}")
n = t.numpy()
print(f"n: {n}")

t: tensor([1., 1., 1., 1., 1.])
n: [1. 1. 1. 1. 1.]

A change in the tensor reflects in the Numpy array.

t.add_(1)
print(f"t: {t}")
print(f"n: {n}")

t: tensor([2., 2., 2., 2., 2.])
n: [2. 2. 2. 2. 2.]

Numpy array to Tensor

n = np.ones(5)
t = torch.from_numpy(n)

Changes in the Numpy array reflects in the tensor.

np.add(n, 1, out=n)
print(f"t: {t}")
print(f"n: {n}")

t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)
n: [2. 2. 2. 2. 2.]