Can i use it for modified BERT with pytorch

[talhaanwarch]

Can i use it for following model

bertmodel=AutoModel.from_pretrained(model_name,output_hidden_states=True)
fc1 = torch.nn.Linear(768, 512)
fc2 = torch.nn.Linear(512, 1)
out=bertmodel(input_ids.squeeze(1),attention_mask.squeeze(1))
out=torch.mean(torch.stack([out['hidden_states'][-i] for i in range(1,4)]),dim=[0,2])
x=dropout(out)
x=relu(.fc1(x))
x=fc2(x)

[cdpierse]

Hi @talhaanwarch, thanks for using the package.

It looks like what you are doing in this snippet is adding additional fully connected layers to the BertForSequenceClassification class provided by HuggingFace. This package directly relies on the model being exactly of that class, not an extended subclass. The reason for this is that it can then use the documented properties and attributes of that class to know how to automatically generate explanations.

I would suggest either training and tuning the model's hyperparameters and FC layers using HuggingFaces custom config or possibly trying to use Pytorch Captum directly on your model as this is the explainability package backing this one.

I hope this helps somewhat.