What Is DarkBERT? How a Language Model Was Built to Study the Dark Web
DarkBERT is a research language model built by KAIST and the security company S2W, described in a 2023 paper accepted at ACL 2023. It starts from the RoBERTa base model and was further trained, using masked language modelling, on text crawled from the Tor network.