Matches in SemOpenAlex for { <https://semopenalex.org/work/W3182696977> ?p ?o ?g. }
- W3182696977 abstract "We introduce HTLM, a hyper-text language model trained on a large-scale web crawl. Modeling hyper-text has a number of advantages: (1) it is easily gathered at scale, (2) it provides rich document-level and end-task-adjacent supervision (e.g. class and id attributes often encode document category information), and (3) it allows for new structured prompting that follows the established semantics of HTML (e.g. to do zero-shot summarization by infilling title tags for a webpage that contains the input text). We show that pretraining with a BART-style denoising loss directly on simplified HTML provides highly effective transfer for a wide range of end tasks and supervision levels. HTLM matches or exceeds the performance of comparably sized text-only LMs for zero-shot prompting and fine-tuning for classification benchmarks, while also setting new state-of-the-art performance levels for zero-shot summarization. We also find that hyper-text prompts provide more value to HTLM, in terms of data efficiency, than plain text prompts do for existing LMs, and that HTLM is highly effective at auto-prompting itself, by simply generating the most likely hyper-text formatting for any available training data. We will release all code and models to support future HTLM research." @default.
- W3182696977 created "2021-07-19" @default.
- W3182696977 creator A5004412943 @default.
- W3182696977 creator A5013308036 @default.
- W3182696977 creator A5022671092 @default.
- W3182696977 creator A5024230879 @default.
- W3182696977 creator A5062471396 @default.
- W3182696977 creator A5067919401 @default.
- W3182696977 creator A5081690204 @default.
- W3182696977 date "2021-07-14" @default.
- W3182696977 modified "2023-10-01" @default.
- W3182696977 title "HTLM: Hyper-Text Pre-Training and Prompting of Language Models." @default.
- W3182696977 cites W1522301498 @default.
- W3182696977 cites W1544827683 @default.
- W3182696977 cites W1599016936 @default.
- W3182696977 cites W1763968285 @default.
- W3182696977 cites W1956340063 @default.
- W3182696977 cites W2101105183 @default.
- W3182696977 cites W2133512280 @default.
- W3182696977 cites W2149327368 @default.
- W3182696977 cites W2154652894 @default.
- W3182696977 cites W2293778248 @default.
- W3182696977 cites W2396767181 @default.
- W3182696977 cites W2563351168 @default.
- W3182696977 cites W2732004306 @default.
- W3182696977 cites W2786660442 @default.
- W3182696977 cites W2888482885 @default.
- W3182696977 cites W2899386490 @default.
- W3182696977 cites W2946659172 @default.
- W3182696977 cites W2950700477 @default.
- W3182696977 cites W2956105246 @default.
- W3182696977 cites W2963310665 @default.
- W3182696977 cites W2963846996 @default.
- W3182696977 cites W2965373594 @default.
- W3182696977 cites W2981852735 @default.
- W3182696977 cites W2982399380 @default.
- W3182696977 cites W2983040767 @default.
- W3182696977 cites W2996264288 @default.
- W3182696977 cites W3015468748 @default.
- W3182696977 cites W3026404337 @default.
- W3182696977 cites W3030163527 @default.
- W3182696977 cites W3033529678 @default.
- W3182696977 cites W3039127676 @default.
- W3182696977 cites W3085177480 @default.
- W3182696977 cites W3091156754 @default.
- W3182696977 cites W3105238007 @default.
- W3182696977 cites W3118216348 @default.
- W3182696977 cites W3119438769 @default.
- W3182696977 cites W3127622310 @default.
- W3182696977 cites W3137573489 @default.
- W3182696977 cites W3164972323 @default.
- W3182696977 cites W3167602185 @default.
- W3182696977 cites W3170305303 @default.
- W3182696977 hasPublicationYear "2021" @default.
- W3182696977 type Work @default.
- W3182696977 sameAs 3182696977 @default.
- W3182696977 citedByCount "4" @default.
- W3182696977 countsByYear W31826969772021 @default.
- W3182696977 countsByYear W31826969772022 @default.
- W3182696977 crossrefType "posted-content" @default.
- W3182696977 hasAuthorship W3182696977A5004412943 @default.
- W3182696977 hasAuthorship W3182696977A5013308036 @default.
- W3182696977 hasAuthorship W3182696977A5022671092 @default.
- W3182696977 hasAuthorship W3182696977A5024230879 @default.
- W3182696977 hasAuthorship W3182696977A5062471396 @default.
- W3182696977 hasAuthorship W3182696977A5067919401 @default.
- W3182696977 hasAuthorship W3182696977A5081690204 @default.
- W3182696977 hasConcept C111919701 @default.
- W3182696977 hasConcept C136764020 @default.
- W3182696977 hasConcept C137293760 @default.
- W3182696977 hasConcept C148730421 @default.
- W3182696977 hasConcept C154945302 @default.
- W3182696977 hasConcept C162324750 @default.
- W3182696977 hasConcept C170858558 @default.
- W3182696977 hasConcept C177264268 @default.
- W3182696977 hasConcept C184337299 @default.
- W3182696977 hasConcept C187736073 @default.
- W3182696977 hasConcept C199360897 @default.
- W3182696977 hasConcept C204321447 @default.
- W3182696977 hasConcept C21959979 @default.
- W3182696977 hasConcept C23123220 @default.
- W3182696977 hasConcept C2776760102 @default.
- W3182696977 hasConcept C2777212361 @default.
- W3182696977 hasConcept C2780451532 @default.
- W3182696977 hasConcept C41008148 @default.
- W3182696977 hasConcept C46503548 @default.
- W3182696977 hasConcept C88006597 @default.
- W3182696977 hasConceptScore W3182696977C111919701 @default.
- W3182696977 hasConceptScore W3182696977C136764020 @default.
- W3182696977 hasConceptScore W3182696977C137293760 @default.
- W3182696977 hasConceptScore W3182696977C148730421 @default.
- W3182696977 hasConceptScore W3182696977C154945302 @default.
- W3182696977 hasConceptScore W3182696977C162324750 @default.
- W3182696977 hasConceptScore W3182696977C170858558 @default.
- W3182696977 hasConceptScore W3182696977C177264268 @default.
- W3182696977 hasConceptScore W3182696977C184337299 @default.
- W3182696977 hasConceptScore W3182696977C187736073 @default.
- W3182696977 hasConceptScore W3182696977C199360897 @default.
- W3182696977 hasConceptScore W3182696977C204321447 @default.
- W3182696977 hasConceptScore W3182696977C21959979 @default.