Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-opus-4-20250514-thinking-16k· AnthropicHugging Face LMArena Human Preference 140K

fix the grammar I have trained the baseline model on the CelebAHQ-Ref dataset (~6500…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.5-flash-preview-04-17’s answer.
  1. User

    fix the grammar I have trained the baseline model on the CelebAHQ-Ref dataset (~6500 training data) for about 600 epochs and ~500K iterations for the restoration task. Baseline: Using pre-trained Codebook and Decoder, Fine-tuning Encoder and training the Transformer

    I have attached the results. The thing is that the loss hasn’t decreased properly. We have two losses 1) MSE loss to encourage the LQ feature to closely resemble the quantized feature and 2) the Cross enropy loss for code index prediction. Most of the pressure of loss decreasing is on the MSE loss. This one doesn’t decrease properly, so the gap between the LQ feature and HQ feature is not mitigated. So, the Transformer fail to match the correct indices.

    If you train the baseline on the HQ images as a sanity check, it converges very fast. I have also checked the gradient flow, and we had gradient for the Transformer and the Encoder during training. As I sent an update before, model can restore one image very well, but when it comes to a dataset it fails.

    One thing that may be cause, there are a few codes that are related to the faces. Or since we have downsampling factor of 8, 4096 tokens are too much for transformer compared to the codeformer which is 256 tokens. Cede embeddings of 4 is too small. The other thing maybe increasing the noise level step by step during the training helps. Otherwise, we need another backbone, or we if we care about the downsampling factor, which 16 might be a better case for us, we need to train the custom model on the FFHQ dataset.

    The other thing is that we can take the codeformer to take LQ images and predict the codes and take the LDM VQVAE model to take reference images and find the matched codes (since it is better than codeformer for HQ reconstruction), we then need to train a fusion module to fuse the outputs of these two which comes before decoder. We also need to fine tune the decoder as well.

    Also, there was another model on Taming-Transformer GitHub which was trained on the FFHQ dataset with the size of 256x256 and the codebook size was 1024 with the downsampling factor of 16. But the reconstruction quality wasn’t desired for the size of 512x512 that we need.

  2. Claude Assistant

    Here's the text with corrected grammar:

    I have trained the baseline model on the CelebAHQ-Ref dataset (~6,500 training samples) for about 600 epochs and ~500K iterations for the restoration task.

    Baseline: Using pre-trained Codebook and Decoder, Fine-tuning Encoder and training the Transformer

    I have attached the results. The issue is that the loss hasn't decreased properly. We have two losses: 1) MSE loss to encourage the LQ feature to closely resemble the quantized feature, and 2) the cross-entropy loss for code index prediction. Most of the burden for decreasing the loss falls on the MSE loss. This loss doesn't decrease properly, so the gap between the LQ feature and HQ feature is not mitigated. Therefore, the Transformer fails to match the correct indices.

    If you train the baseline on the HQ images as a sanity check, it converges very quickly.

    I have also checked the gradient flow, and we had gradients for the Transformer and the Encoder during training. As I mentioned in a previous update, the model can restore one image very well, but when it comes to a dataset, it fails.

    One thing that may be the cause is that there are only a few codes related to faces. Or since we have a downsampling factor of 8, 4,096 tokens may be too many for the transformer compared to CodeFormer, which has 256 tokens. Code embeddings of 4 are too small. Another possibility is that increasing the noise level step by step during training might help. Otherwise, we need another backbone, or if we care about the downsampling factor, in which case 16 might be better for us, we need to train the custom model on the FFHQ dataset.

    Another option is to use CodeFormer to process LQ images and predict the codes, and use the LDM VQVAE model to process reference images and find the matched codes (since it is better than CodeFormer for HQ reconstruction). We would then need to train a fusion module to combine the outputs of these two models, which would come before the decoder. We would also need to fine-tune the decoder.

    Additionally, there was another model on the Taming-Transformer GitHub repository that was trained on the FFHQ dataset at a resolution of 256×256 with a codebook size of 1,024 and a downsampling factor of 16. However, the reconstruction quality wasn't satisfactory for the 512×512 resolution that we need.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.