4000-520-616
欢迎来到免疫在线!(蚂蚁淘生物旗下平台)  请登录 |  免费注册 |  询价篮
主营:原厂直采,平行进口,授权代理(蚂蚁淘为您服务)
咨询热线电话
4000-520-616
当前位置: 首页 > 新闻动态 >
新闻详情
018 tensorflow | nce loss
来自 : www.fashici.com/tech/9...html 发布时间:2021-03-25
train_inputs = tf.placeholder(tf.int32, shape=[batch_size])train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])embeddings = tf.Variable( tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))embed = tf.nn.embedding_lookup(embeddings, train_inputs)def nce_loss(weights, biases, labels, inputs, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=False, partition_strategy= mod , name= nce_loss )
weights: A `Tensor` of shape `[num_classes, dim]`, or a list of `Tensor` objects whose concatenation along dimension 0 has shape [num_classes, dim]. The (possibly-partitioned) class embeddings. biases: A `Tensor` of shape `[num_classes]`. The class biases. labels: A `Tensor` of type `int64` and shape `[batch_size, num_true]`. The target classes. inputs: A `Tensor` of shape `[batch_size, dim]`. The forward activations of the input network.
nce loss计算流程负样本采样num_samples - input vector分别与(num_samples + 1)个向量做内积,得到预测值x,x 与y做交叉熵loss(此时维度为batch_size * (num_samples + 1))然后用ones矩阵和loss矩阵相乘,得到batch_size * 1的 loss向量,然后求均值
def log_uniform_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None):

所以,默认情况下,他会用log_uniform_candidate_sampler去采样。那么log_uniform_candidate_sampler是怎么采样的呢?他的实现在这里:

他会在[0, range_max)中采样出一个整数kP(k) = (log(k + 2) - log(k + 1)) / log(range_max + 1)

关于word2vec的skip-gram模型使用负例采样nce_loss损失函数的源码剖析
2020年1月14日nn.embedding_lookup(embeddings, train_inputs) def nce_loss(weights, biases, labels, inputs, num_sampled, num_classes, num_true=1, sampled_va...tensorflow,远鸿博客018 tensorflow | nce loss018 tensorflow

本文链接: http://nceed.immuno-online.com/view-747321.html

发布于 : 2021-03-25 阅读(0)