AI model inference training

phixtral-2x2_8 is the first hybrid expert model made from two Microsoft/Phi-2 models, inspired by the Mistralai/Mixtral-8x7B-v0.1 architecture. Its performance is better than each individual expert model. The model performs well on multiple evaluation indicators such as AGIEval, GPT4All, TruthfulQA, Bigbench, etc. It uses a custom version of the mergekit library (mixtral branch) and specific configuration. Users can run Phixtral on a free T4 GPU in a Colab notebook with 4-digit precision. The model size is 4.46B parameters and the tensor type is F16.