锐客网

  1. 首页 > 睿知 > it技术 > >

Tensorflow中循环神经网络及其Wrappers

IT知识神经网络tensorflow

宝剑锋从磨砺出，梅花香自苦寒来。这篇文章主要讲述Tensorflow中循环神经网络及其Wrappers相关的知识，希望能为你提供帮助。

  • tf.nn.rnn_cell.LSTMCell
    • 又名：tf.nn.rnn_cell.BasicLSTMCell、tf.contrib.rnn.LSTMCell
    • 参见: tf.nn.rnn_cell.LSTMCell
    • 输出：
      • output：LSTM单元输出，与LSTM cell state的区别在于该输出又经过激活以及和一个sigmoid函数输出相乘。shape: [batch_size,num_units]
      • new_state：当前时间步上的LSTM cell state和LSTM output。使用数据结构LSTMStateTuple描述，LSTMStateTuple：(c,h)，其中，h与上述的output完全相同。shape: ([batch_size,num_units],[batch_size,num_units])
    • 示例：
      batch_size=10 embedding_size=300 inputs=tf.Variable(tf.random_normal([batch_size,embedding_size])) previous_state=(tf.Variable(tf.random_normal([batch_size,128])),tf.Variable(tf.random_normal([batch_size,128]))) lstmcell=tf.nn.rnn_cell.LSTMCell(128) outputs,states=lstmcell(inputs,previous_state)

      输出：
      outputs: < tf.Tensor ‘lstm_cell/mul_2:0‘ shape=(10, 128) dtype=float32> states: LSTMStateTuple(c=< tf.Tensor ‘lstm_cell/add_1:0‘ shape=(10, 128) dtype=float32> , h=< tf.Tensor ‘lstm_cell/mul_2:0‘ shape=(10, 128) dtype=float32> )

  • tf.nn.rnn_cell.MultiRNNCell
    • 参见：tf.nn.rnn_cell.MultiRNNCell
    • 输出：
      • outputs: 最顶层cell的最后一个时间步的输出。shape：[batch_size,cell.output_size]
      • states：每一层的state，M层LSTM则输出M个LSTMStateTuple组成的Tuple。
    • 示例：
      batch_size=10 inputs=tf.Variable(tf.random_normal([batch_size,128])) previous_state0=(tf.random_normal([batch_size,100]),tf.random_normal([batch_size,100])) previous_state1=(tf.random_normal([batch_size,200]),tf.random_normal([batch_size,200])) previous_state2=(tf.random_normal([batch_size,300]),tf.random_normal([batch_size,300])) num_units=[100,200,300] cells=[tf.nn.rnn_cell.LSTMCell(num_unit) for num_unit in num_units] mul_cells=tf.nn.rnn_cell.MultiRNNCell(cells) outputs,states=mul_cells(inputs,(previous_state0,previous_state1,previous_state2))

      输出：
      outputs: < tf.Tensor ‘multi_rnn_cell_1/cell_2/lstm_cell/mul_2:0‘ shape=(10, 300) dtype=float32> states: (LSTMStateTuple(c=< tf.Tensor ‘multi_rnn_cell_1/cell_0/lstm_cell/add_1:0‘ shape=(10, 100) dtype=float32> , h=< tf.Tensor ‘multi_rnn_cell_1/cell_0/lstm_cell/mul_2:0‘ shape=(10, 100) dtype=float32> ), LSTMStateTuple(c=< tf.Tensor ‘multi_rnn_cell_1/cell_1/lstm_cell/add_1:0‘ shape=(10, 200) dtype=float32> , h=< tf.Tensor ‘multi_rnn_cell_1/cell_1/lstm_cell/mul_2:0‘ shape=(10, 200) dtype=float32> ), LSTMStateTuple(c=< tf.Tensor ‘multi_rnn_cell_1/cell_2/lstm_cell/add_1:0‘ shape=(10, 300) dtype=float32> , h=< tf.Tensor ‘multi_rnn_cell_1/cell_2/lstm_cell/mul_2:0‘ shape=(10, 300) dtype=float32> ))

  • tf.nn.dynamic_rnn
    • 参见：tf.nn.dynamic_rnn
    • 输出：
      • outputs: 每个时间步上的LSTM输出；若有多层LSTM，则为每一个时间步上最顶层的LSTM的输出。shape: [batch_size,max_time,cell.output_size]
      • state：最后一个时间步的状态，该状态使用LSTMStateTuple结构输出；若有M层LSTM，则输出M个LSTMStateTuple。shape：单层LSTM输出：[batch_size,cell.output_size]；M层LSTM输出：M个LSTMStateTuple组成的Tuple，这也即是说：outputs[:,-1,:]==state[-1,:,:]。
    • 示例：
      batch_size=10 max_time=20 data=https://www.songbingjia.com/android/tf.Variable(tf.random_normal([batch_size,max_time,128])) # create a BasicRNNCell rnn_cell = tf.nn.rnn_cell.BasicRNNCell(num_units=128)# defining initial state initial_state = rnn_cell.zero_state(batch_size,dtype=tf.float32)# ‘outputs‘ is a tensor of shape [batch_size, max_time, cell_state_size] # ‘state‘ is a tensor of shape [batch_size, cell_state_size] outputs, state = tf.nn.dynamic_rnn(cell=rnn_cell, inputs=data, initial_state=initial_state, dtype=tf.float32)

      输出：
      outpus: < tf.Tensor ‘rnn_2/transpose_1:0‘ shape=(10, 20, 128) dtype=float32> state: < tf.Tensor ‘rnn_2/while/Exit_3:0‘ shape=(10, 128) dtype=float32>

      batch_size=10 max_time=20 data=https://www.songbingjia.com/android/tf.Variable(tf.random_normal([batch_size,max_time,128])) # create 2 LSTMCells rnn_layers = [tf.nn.rnn_cell.LSTMCell(size) for size in [128, 256]]# create a RNN cell composed sequentially of a number of RNNCells multi_rnn_cell = tf.nn.rnn_cell.MultiRNNCell(rnn_layers)# ‘outputs‘ is a tensor of shape [batch_size, max_time, 256] # ‘state‘ is a N-tuple where N is the number of LSTMCells containing a # tf.contrib.rnn.LSTMStateTuple for each cell outputs, state = tf.nn.dynamic_rnn(cell=multi_rnn_cell, inputs=data, dtype=tf.float32)

      outputs: < tf.Tensor ‘rnn_1/transpose_1:0‘ shape=(10, 20, 256) dtype=float32> state: (LSTMStateTuple(c=< tf.Tensor ‘rnn_1/while/Exit_3:0‘ shape=(10, 128) dtype=float32> , h=< tf.Tensor ‘rnn_1/while/Exit_4:0‘ shape=(10, 128) dtype=float32> ), LSTMStateTuple(c=< tf.Tensor ‘rnn_1/while/Exit_5:0‘ shape=(10, 256) dtype=float32> , h=< tf.Tensor ‘rnn_1/while/Exit_6:0‘ shape=(10, 256) dtype=float32> ))

  • tf.nn.bidirectional_dynamic_rnn
    • 参见：tf.nn.bidirectional_dynamic_rnn
    • 输出：
      • outputs：(output_fw,output_bw)：前向cell+后向cell
        其中，output_fw、output_bw均为：[batch_size,max_time,cell.output_size]
      • state：(output_state_fw,output_state_bw)：包含前向和后向隐状态组成的元组
        其中，output_state_fw、output_state_bw均为LSTMStateTuple。LSTMStateTuple:(c,h)，分别为cell_state，hidden_output
  • 【Tensorflow中循环神经网络及其Wrappers】tf.contrib.seq2seq.dynamic_decode
    • 输出：
      • final_outputs，包含rnn_output和sample_id，分别可用final_output.rnn_output和final_outputs.sample_id获取到。
      • final_state，可以从最后一个解码器状态获取alignments，alignments = tf.transpose(final_decoder_state.alignment_history.stack(), [1, 2, 0])
      • final_sequence_lengths

    推荐阅读


    上一篇：Android 自定义倾斜字体

    下一篇：Flutter - 本地化启动列表中App名字