EmbeddingsToTextHelper Module

Provides functions to convert embeddings into human-readable text and related utilities.

class HMB.EmbeddingsToTextHelper.EmbeddingsToTextModel(tokenizeModelName='t5-small', inputFeatureDim=6144, hiddenDim=512, generationMaxLength=512, dropoutRatio=0.1, numPromptTokens=5)

Bases: Module

A PyTorch model that generates text from input features using a pre-trained T5 model. The model includes a feature projection layer to transform input features into a suitable format for text generation. It also incorporates learnable prompt tokens to enhance the generation process.

Variables:

• t5 (transformers.T5ForConditionalGeneration) – Pre-trained T5 model for text generation.

• tokenizer (transformers.T5Tokenizer) – Tokenizer for the T5 model.

• featureProjection (torch.nn.Sequential) – Sequential layer for projecting input features.

• toT5Hidden (torch.nn.Linear) – Linear layer to project features to T5’s hidden size.

• promptEmbeddings (torch.nn.Parameter) – Learnable prompt embeddings for the model.

• numPromptTokens (int) – Number of learnable prompt tokens.

• generationMaxLength (int) – Maximum length for generated text sequences.

Examples

import HMB.EmbeddingsToTextHelper as e2tt

# Initialize the model with default parameters.
model = e2tt.EmbeddingsToTextModel(
  tokenizeModelName="t5-small",
  inputFeatureDim=6144,
  hiddenDim=512,
  generationMaxLength=512,
  dropoutRatio=0.1,
  numPromptTokens=5
)

# Print the model architecture.
print(model)
# Example input features (batch size of 2, feature dimension of 6144).
exampleFeatures = torch.randn(2, 6144)
# Generate text from the example features.
generatedIds = model.generate(exampleFeatures, max_length=50)
# Decode the generated text to a human-readable format.
generatedText = model.tokenizer.batch_decode(generatedIds, skip_special_tokens=True)
print(generatedText)

Initialize the EmbeddingsToTextModel for generating text from features. This model uses a pre-trained T5 model and adds a feature projection layer to transform input features into a format suitable for text generation.

Parameters:

• tokenizeModelName (str) – Name of the pre-trained T5 model to use (default: “t5-small”).

• inputFeatureDim (int) – Dimension of the input feature vector (default: 6144).

• hiddenDim (int) – Hidden dimension for the feature projection layers (default: 512).

• generationMaxLength (int) – Maximum length for the generated text (default: 512).

• dropoutRatio (float) – Dropout ratio for regularization (default: 0.1).

• numPromptTokens (int) – Number of learnable prompt tokens (default: 5).

__init__(tokenizeModelName='t5-small', inputFeatureDim=6144, hiddenDim=512, generationMaxLength=512, dropoutRatio=0.1, numPromptTokens=5)

Initialize the EmbeddingsToTextModel for generating text from features. This model uses a pre-trained T5 model and adds a feature projection layer to transform input features into a format suitable for text generation.

Parameters:

• tokenizeModelName (str) – Name of the pre-trained T5 model to use (default: “t5-small”).

• inputFeatureDim (int) – Dimension of the input feature vector (default: 6144).

• hiddenDim (int) – Hidden dimension for the feature projection layers (default: 512).

• generationMaxLength (int) – Maximum length for the generated text (default: 512).

• dropoutRatio (float) – Dropout ratio for regularization (default: 0.1).

• numPromptTokens (int) – Number of learnable prompt tokens (default: 5).

forward(features, input_ids=None, attention_mask=None, labels=None)

Forward pass through the EmbeddingsToTextModel. This method processes the input features, projects them to T5’s hidden dimension, and generates text using the T5 model.

Parameters:

• features (torch.Tensor) – Input features to be projected and processed.

• input_ids (torch.Tensor, optional) – Input token IDs for the T5 model (default: None).

• attentionMask (torch.Tensor, optional) – Attention mask to indicate which tokens are valid (default: None).

• labels (torch.Tensor, optional) – Labels for the causal language modeling task (default: None).

Returns:

Output from the T5 model containing logits and loss. It includes the generated text and loss if labels are provided.

Return type:

transformers.modeling_outputs.Seq2SeqLMOutput

generate(features, **kwargs)

Generate text from input features using the T5 model. This method projects the input features and generates text based on the provided parameters.

Parameters:

• features (torch.Tensor) – Input features to be projected and processed.

• **kwargs – Additional keyword arguments for the generation method.

Returns:

Generated text token IDs.

Return type:

torch.Tensor

HMB.EmbeddingsToTextHelper.TrainModel(model, trainLoader, valLoader, numEpochs=10, learningRate=0.0001, optimizerType='adamw', modelStoragePath='BestModel.pth', verbose=False)

Train the EmbeddingsToTextModel using the provided training and validation data loaders. This function performs the training loop, including forward and backward passes, loss computation, and optimization steps. It also evaluates the model on the validation set after each epoch to monitor performance and saves the best model state based on validation loss.

Parameters:

• model (EmbeddingsToTextModel) – Instance of the EmbeddingsToTextModel to be trained.

• trainLoader (DataLoader) – DataLoader for training data.

• valLoader (DataLoader) – DataLoader for validation data.

• numEpochs (int) – Number of epochs to train the model (default: 10).

• learningRate (float) – Learning rate for the optimizer (default: 1e-4).

• optimizerType (str) – Type of optimizer to use for training (default: “adamw”).

• modelStoragePath (str) – Path to save the best model state (default: “BestModel.pth”).

• verbose (bool) – Whether to print verbose output during training (default: False).

Examples

import HMB.EmbeddingsToTextHelper as e2tt

# Initialize the model with default parameters.
model = e2tt.EmbeddingsToTextModel(
  tokenizeModelName="t5-small",
  inputFeatureDim=6144,
  hiddenDim=512,
  generationMaxLength=512,
  dropoutRatio=0.1,
  numPromptTokens=5,
)
# Assume trainLoader and valLoader are predefined DataLoader instances.
trainLoader = ...  # Your training DataLoader here.
valLoader = ...    # Your validation DataLoader here.
# Print the model architecture.
print(model)

# Train the model using the training and validation data loaders.
e2tt.TrainModel(
  model=model,
  trainLoader=trainLoader,
  valLoader=valLoader,
  numEpochs=10,
  learningRate=1e-4,
  optimizerType="adamw",
  modelStoragePath="BestModel.pth",
  verbose=True
)