Matches in SemOpenAlex for { <https://semopenalex.org/work/W4307656139> ?p ?o ?g. }
Showing items 1 to 54 of
54
with 100 items per page.
- W4307656139 abstract "Transformer models have been developed in molecular science with excellent performance in applications including quantitative structure-activity relationship (QSAR) and virtual screening (VS). Compared with other types of models, however, they are large, which results in a high hardware requirement to abridge time for both training and inference processes. In this work, cross-layer parameter sharing (CLPS), and knowledge distillation (KD) are used to reduce the sizes of transformers in molecular science. Both methods not only have competitive QSAR predictive performance as compared to the original BERT model, but also are more parameter efficient. Furthermore, by integrating CLPS and KD into a two-state chemical network, we introduce a new deep lite chemical transformer model, DeLiCaTe. DeLiCaTe captures general-domains as well as task-specific knowledge, which lead to a 4x faster rate of both training and inference due to a 10- and 3-times reduction of the number of parameters and layers, respectively. Meanwhile, it achieves comparable performance in QSAR and VS modeling. Moreover, we anticipate that the model compression strategy provides a pathway to the creation of effective generative transformer models for organic drug and material design." @default.
- W4307656139 created "2022-11-04" @default.
- W4307656139 creator A5052676364 @default.
- W4307656139 creator A5057304913 @default.
- W4307656139 date "2022-05-16" @default.
- W4307656139 modified "2023-09-23" @default.
- W4307656139 title "Chemical transformer compression for accelerating both training and inference of molecular modeling" @default.
- W4307656139 doi "https://doi.org/10.48550/arxiv.2205.07582" @default.
- W4307656139 hasPublicationYear "2022" @default.
- W4307656139 type Work @default.
- W4307656139 citedByCount "0" @default.
- W4307656139 crossrefType "posted-content" @default.
- W4307656139 hasAuthorship W4307656139A5052676364 @default.
- W4307656139 hasAuthorship W4307656139A5057304913 @default.
- W4307656139 hasBestOaLocation W43076561391 @default.
- W4307656139 hasConcept C119599485 @default.
- W4307656139 hasConcept C119857082 @default.
- W4307656139 hasConcept C127413603 @default.
- W4307656139 hasConcept C154945302 @default.
- W4307656139 hasConcept C164126121 @default.
- W4307656139 hasConcept C165801399 @default.
- W4307656139 hasConcept C2776214188 @default.
- W4307656139 hasConcept C39890363 @default.
- W4307656139 hasConcept C41008148 @default.
- W4307656139 hasConcept C51632099 @default.
- W4307656139 hasConcept C66322947 @default.
- W4307656139 hasConceptScore W4307656139C119599485 @default.
- W4307656139 hasConceptScore W4307656139C119857082 @default.
- W4307656139 hasConceptScore W4307656139C127413603 @default.
- W4307656139 hasConceptScore W4307656139C154945302 @default.
- W4307656139 hasConceptScore W4307656139C164126121 @default.
- W4307656139 hasConceptScore W4307656139C165801399 @default.
- W4307656139 hasConceptScore W4307656139C2776214188 @default.
- W4307656139 hasConceptScore W4307656139C39890363 @default.
- W4307656139 hasConceptScore W4307656139C41008148 @default.
- W4307656139 hasConceptScore W4307656139C51632099 @default.
- W4307656139 hasConceptScore W4307656139C66322947 @default.
- W4307656139 hasLocation W43076561391 @default.
- W4307656139 hasLocation W43076561392 @default.
- W4307656139 hasOpenAccess W4307656139 @default.
- W4307656139 hasPrimaryLocation W43076561391 @default.
- W4307656139 hasRelatedWork W2799803467 @default.
- W4307656139 hasRelatedWork W2963058055 @default.
- W4307656139 hasRelatedWork W3030978062 @default.
- W4307656139 hasRelatedWork W3094491777 @default.
- W4307656139 hasRelatedWork W3117501106 @default.
- W4307656139 hasRelatedWork W3214715529 @default.
- W4307656139 hasRelatedWork W4206178588 @default.
- W4307656139 hasRelatedWork W4280500920 @default.
- W4307656139 hasRelatedWork W4287635093 @default.
- W4307656139 hasRelatedWork W4307656139 @default.
- W4307656139 isParatext "false" @default.
- W4307656139 isRetracted "false" @default.
- W4307656139 workType "article" @default.