Hi All! Sorry in advance for the incomplete links. I’m a new user and the system wouldn’t let me have them, but i still wanted to reference what i was looking at, so both for OpenNMT-py and pyTorch, just prepend github.com/

I’m trying to build a custom Loss function to use on an OpenNMT-py model. I’ve seen this post where there are some references for the Lua implementation, but i’m finding i cannot follow through for python.

If i look at the Loss module @ OpenNMT/OpenNMT-py/blob/master/onmt/Loss.py , it seems that since i don’t need to change the overall logic, i would only need to change the crit input to that object. In order to do so, i’ve looked into the MNTCriterion @ OpenNMT/OpenNMT-py/blob/master/onmt/Loss.py#L18 to use as a template.

Now is where my troubles begin:

def forward(self, input, target):
        _assert_no_grad(target)
        return F.nll_loss(input, target, self.weight, self.size_average,
                          self.ignore_index)
Now i try to understand what F.nll_loss @ pytorch/pytorch/blob/master/torch/nn/modules/loss.py#L131 does, so i go to it’s declaration @ pytorch/pytorch/blob/master/torch/nn/functional.py#L664:
def nll_loss(input, target, weight=None, size_average=True, ignore_index=-100):
    r"""The negative log likelihood loss.
    See :class:`~torch.nn.NLLLoss` for details.
    Args:
        input: :math:`(N, C)` where `C = number of classes` or `(N, C, H, W)`
            in case of 2D - Loss
        target: :math:`(N)` where each value is `0 <= targets[i] <= C-1`
        weight (Variable, optional): a manual rescaling weight given to each
            class. If given, has to be a Variable of size "nclasses"
        size_average (bool, optional): By default, the losses are averaged
            over observations for each minibatch. If size_average
            is False, the losses are summed for each minibatch. Default: True
        ignore_index (int, optional): Specifies a target value that is ignored
            and does not contribute to the input gradient. When size_average is
            True, the loss is averaged over non-ignored targets. Default: -100
    Example::
        >>> # input is of size nBatch x nClasses = 3 x 5
        >>> input = autograd.Variable(torch.randn(3, 5))
        >>> # each element in target has to have 0 <= value < nclasses
        >>> target = autograd.Variable(torch.LongTensor([1, 0, 4]))
        >>> output = F.nll_loss(F.log_softmax(input), target)
        >>> output.backward()
    dim = input.dim()
    if dim == 2:
        return _functions.thnn.NLLLoss.apply(input, target, weight, size_average, ignore_index)
    elif dim == 4:
        return _functions.thnn.NLLLoss2d.apply(input, target, weight, size_average, ignore_index)
    else:
        raise ValueError('Expected 2 or 4 dimensions (got {})'.format(dim))
and for the life of me, i know not how to read what that does. I can try to drill down on my IDE, or step through on debug mode, but that only gets worse, as i arrive to something like this:
@staticmethod
    def forward(ctx, input, target, *args):
        ctx._backend = type2backend[type(input)]
        ctx.save_for_backward(input, target)
        if weight_arg_idx >= 0:
            ctx.weight = args[0]
            args = args[1:]
            ctx.additional_args = list(args)
            insert_idx = weight_arg_idx - 4  # state, input, target, output
            ctx.additional_args.insert(insert_idx, ctx.weight)
        else:
            ctx.additional_args = list(args)
        ctx.forward_args_count = len(ctx.additional_args)
        for idx in buffers_idx:
            ctx.additional_args.insert(idx, input.new(1))
        output = input.new(1)
        getattr(ctx._backend, update_output.name)(ctx._backend.library_state, input, target,
                                                  output, *ctx.additional_args)
        return output
All i want to do is find the template from which the criterion is effectively calculated so i can create a modified copy that suits my needs.
Anyone have an idea of where and how would this be done?
Thanks!
But the links are outdated. I could find this on an older branch: https://github.com/pytorch/pytorch/blob/v0.3.1/torch/lib/THNN/generic/ClassNLLCriterion.c
If you  find the new location, please comment either here or on the pytorch thread!