AI Definitions: Distillation
/Distillation (also called knowledge distillation) – This is training an AI model using the outputs of another model. Small models can learn from larger teacher models. There is disagreement as to where to draw the line between using this technique for the legitimate study of how AI models work and unethical business practices. It is yet to be determined whether it violates any U.S law. Many Chinese AI models have been accused of using it to accelerate the development of their own models through overseas proxy services called transfer stations. Knowledge distillation was first discussed in a 2015 paper by three Google researchers. It will probably fade in importance as AI companies move toward developing AI agents.