原先在DIN中User Behaviors的部分现在分成了三层，第一层Behavior Layer，第二层Interest Extractor Layer，第三层Interest Evolving Layer。
- 将用户行为序列embedding之后和其他特征embedding一起作为输入
- 兴趣抽取层
Interest Extractor Layer：用户行为序列送入GRU结构，使用t时刻及之前的序列抽取t时刻的兴趣（即GRU对应细胞的输出h t h_t h t ） - 在兴趣抽取层中引入辅助loss，最大化h t h_t h t 与t+1时刻item embedding的内积，最小化h t h_t h t 与随机采样负例的内积
- 辅助loss：对内积进行sigmoid转化为0~1之间的值之后，再与1/0得到binary交叉熵，与graphsage非监督学习的loss形式一致，在tf中实现为sigmoid_cross_entropy_with_logits
- 辅助loss能够对GRU的每一步进行学习，一方面有利于GRU的学习，使模型能够好地抓住某一时刻的兴趣点，另一方面也有利于item embedding的学习
- 将兴趣抽取层GRU的输出与目标向量e做attention，得到attention权重，具体做法是对h t W e h_t W_e h t W e 进行softmax，其中W是待学习的权重参数
- 兴趣进化层
Interest Evolving Layer：在兴趣抽取层之上再堆叠一层GRU，使用第一层GRU的输出作为输入，与第一层GRU的的区别在于更新门的输出u需要乘上attention权重，通过attention机制控制每一步GRU隐含向量的更新 - 将兴趣进化层GRU最后一步的输出与其他特征embedding做concat，送入多层简单DNN拟合真实click
详见：
https://blog.csdn.net/wuzhongqiang/article/details/109532438
import os, warnings, time, sys
import pickle
import matplotlib.pyplot as plt
import pandas as pd, numpy as np
from sklearn.utils import shuffle
from sklearn.metrics import f1_score, accuracy_score, roc_curve, precision_score, recall_score, roc_auc_score
from sklearn import metrics
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler, OneHotEncoder
from sklearn.preprocessing import LabelEncoder
from deepctr.models import DIEN
from deepctr.feature_column import SparseFeat, DenseFeat, get_feature_names
from deepctr.layers import custom_objects
from tensorflow.keras.models import model_from_yaml
import tensorflow as tf
from tensorflow.python.ops import array_ops
import tensorflow.keras.backend as K
from sklearn import datasets
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical
from keras.models import model_from_json
from tensorflow.keras.callbacks import *
from tensorflow.keras.models import *
from tensorflow.keras.layers import *
from tensorflow.keras.optimizers import *
from keras.preprocessing.sequence import pad_sequences
from keras.preprocessing.text import one_hot
from keras.layers.embeddings import Embedding
from deepctr.feature_column import SparseFeat, VarLenSparseFeat, DenseFeat, get_feature_names
from toolsnn import *
import settings
def get_xy_fd(use_neg=False, hash_flag=False):
behavior_feature_list = ["item_id", "cate_id"]
uid = np.array([0, 1, 2])
ugender = np.array([0, 1, 0])
iid = np.array([1, 2, 3])
cate_id = np.array([1, 2, 2])
score = np.array([0.1, 0.2, 0.3])
hist_iid = np.array([[1, 2, 3, 0], [1, 2, 3, 0], [1, 2, 0, 0]])
hist_cate_id = np.array([[1, 2, 2, 0], [1, 2, 2, 0], [1, 2, 0, 0]])
behavior_length = np.array([3, 3, 2])
feature_columns = [SparseFeat('user', 3, embedding_dim=10, use_hash=hash_flag),
SparseFeat('gender', 2, embedding_dim=4, use_hash=hash_flag),
SparseFeat('item_id', 3 + 1, embedding_dim=8, use_hash=hash_flag),
SparseFeat('cate_id', 2 + 1, embedding_dim=4, use_hash=hash_flag),
DenseFeat('pay_score', 1)]
feature_columns += [VarLenSparseFeat(SparseFeat('hist_item_id', vocabulary_size=3 + 1, embedding_dim=8, embedding_name='item_id'), maxlen=4, length_name="seq_length"),
VarLenSparseFeat(SparseFeat('hist_cate_id', 2 + 1, embedding_dim=4, embedding_name='cate_id'), maxlen=4, length_name="seq_length")]
feature_dict = {'user': uid, 'gender': ugender, 'item_id': iid, 'cate_id': cate_id,
'hist_item_id': hist_iid, 'hist_cate_id': hist_cate_id,
'pay_score': score, "seq_length": behavior_length}
x = {name: feature_dict[name] for name in get_feature_names(feature_columns)}
y = np.array([1, 0, 1])
print(x)
print(y)
return x, y, feature_columns, behavior_feature_list
if __name__ == "__main__":
if tf.__version__ >= '2.0.0':
tf.compat.v1.disable_eager_execution()
USE_NEG = True
x, y, feature_columns, behavior_feature_list = get_xy_fd(use_neg=USE_NEG)
model = DIEN(feature_columns, behavior_feature_list,
gru_type="AUGRU",
use_negsampling=False,
dnn_hidden_units=(256, 128, 64),
alpha=1.0,
use_bn=True,
att_hidden_units=(64, 16), att_activation="dice",
l2_reg_dnn=0, l2_reg_embedding=0.00001, dnn_dropout=0.1,
task='binary',
)
model.compile('adam', 'binary_crossentropy',
metrics=['binary_crossentropy'])
history = model.fit(x, y, verbose=1, epochs=10, validation_split=0.5)
Original: https://blog.csdn.net/qq_42363032/article/details/121774193
Author: WGS.
Title: CTR—DIEN原理，及deepctr实现DIEN
原创文章受到原创版权保护。转载请注明出处：https://www.johngo689.com/520716/
转载文章受原作者版权保护。转载请注明原作者出处！