Sequence

  • pad_sequence : 填充序列
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# 定义序列个数、最小长度和最大长度
num_sequences = 5
min_length = 3
max_length = 10

# 定义整数范围（例如，从1到100）
low = 1
high = 9

# 生成长度不均的随机整数序列
random_sequences = []
for _ in range(num_sequences):
    length = torch.randint(min_length, max_length + 1, (1,)).item()
    sequence = torch.randint(low, high, (length,))
    random_sequences.append(sequence)

random_sequences
>>>
[tensor([3, 8, 1, 4]),
 tensor([8, 3, 4, 6]),
 tensor([7, 2, 4, 6, 8, 4, 8, 8]),
 tensor([7, 4, 4, 7, 7, 7, 6, 6]),
 tensor([5, 4, 1, 4, 3])]

# 使用pad_sequence对序列进行填充
padded_sequences = pad_sequence(random_sequences, batch_first=True)
padded_sequences
>>>
tensor([[3, 8, 1, 4, 0, 0, 0, 0],
        [8, 3, 4, 6, 0, 0, 0, 0],
        [7, 2, 4, 6, 8, 4, 8, 8],
        [7, 4, 4, 7, 7, 7, 6, 6],
        [5, 4, 1, 4, 3, 0, 0, 0]])
  • pack_padded_sequence
  • pad_packed_sequence
  • 使用RNN（如LSTM或GRU）处理序列数据时，pack_padded_sequence将填充后的序列数据压缩为紧凑的表示形式，以提高计算效率。
  • 在RNN处理完紧凑的序列后，可能需要将输出转换回填充序列的格式，以便进行后续操作（如解码、分类等）。pad_packed_sequence函数将RNN的输出转换回填充序列格式，同时保留原始序列的长度信息。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
lengths = torch.tensor([len(x) for x in random_sequences])
lengths
>>>
tensor([4, 4, 8, 8, 5])

packed_sequences = pack_padded_sequence(padded_sequences, lengths, batch_first=True, enforce_sorted=False)
packed_sequences
>>>
PackedSequence(data=tensor([7, 7, 5, 3, 8, 2, 4, 4, 8, 3, 4, 4, 1, 1, 4, 6, 7, 4, 4, 6, 8, 7, 3, 4,7, 8, 6, 8, 6]), batch_sizes=tensor([5, 5, 5, 5, 3, 2, 2, 2]), sorted_indices=tensor([2, 3, 4, 0, 1]), unsorted_indices=tensor([3, 4, 0, 1, 2]))

seq_unpacked, lens_unpacked = pad_packed_sequence(packed_sequences, batch_first=True)
seq_unpacked
>>>
tensor([[3, 8, 1, 4, 0, 0, 0, 0],
        [8, 3, 4, 6, 0, 0, 0, 0],
        [7, 2, 4, 6, 8, 4, 8, 8],
        [7, 4, 4, 7, 7, 7, 6, 6],
        [5, 4, 1, 4, 3, 0, 0, 0]])

lens_unpacked
>>>
tensor([4, 4, 8, 8, 5])

torch.nn.utils.rnn.pad_sequence — PyTorch 2.1 documentation

torch.nn.utils.rnn.pad_packed_sequence — PyTorch 2.1 documentation