Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287332019> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4287332019 abstract "In a large class of deep learning models, including vector embedding models such as word and database embeddings, we observe that floating point exponent values cluster around a few unique values, permitting entropy based data compression. Entropy coding compresses fixed-length values with variable-length codes, encoding most probable values with fewer bits. We propose the EFloat compressed floating point number format that uses a variable field boundary between the exponent and significand fields. EFloat uses entropy coding on exponent values and signs to minimize the average width of the exponent and sign fields, while preserving the original FP32 exponent range unchanged. Saved bits become part of the significand field increasing the EFloat numeric precision by 4.3 bits on average compared to other reduced-precision floating point formats. EFloat makes 8-bit and even smaller floats practical without sacrificing the exponent range of a 32-bit floating point representation. We currently use the EFloat format for saving memory capacity and bandwidth consumption of large vector embedding models such as those used for database embeddings. Using the RMS error as metric, we demonstrate that EFloat provides higher accuracy than other floating point formats with equal bit budget. The EF12 format with 12-bit budget has less end-to-end application error than the 16-bit BFloat16. EF16 with 16-bit budget has an RMS-error 17 to 35 times less than BF16 RMS-error for a diverse set of embedding models. When making similarity and dissimilarity queries, using the NDCG ranking metric, EFloat matches the result quality of prior floating point representations with larger bit budgets." @default.
- W4287332019 created "2022-07-25" @default.
- W4287332019 creator A5035394281 @default.
- W4287332019 creator A5062158919 @default.
- W4287332019 creator A5067683700 @default.
- W4287332019 date "2021-02-04" @default.
- W4287332019 modified "2023-10-17" @default.
- W4287332019 title "EFloat: Entropy-coded Floating Point Format for Compressing Vector Embedding Models" @default.
- W4287332019 doi "https://doi.org/10.48550/arxiv.2102.02705" @default.
- W4287332019 hasPublicationYear "2021" @default.
- W4287332019 type Work @default.
- W4287332019 citedByCount "0" @default.
- W4287332019 crossrefType "posted-content" @default.
- W4287332019 hasAuthorship W4287332019A5035394281 @default.
- W4287332019 hasAuthorship W4287332019A5062158919 @default.
- W4287332019 hasAuthorship W4287332019A5067683700 @default.
- W4287332019 hasBestOaLocation W42873320191 @default.
- W4287332019 hasConcept C106301342 @default.
- W4287332019 hasConcept C11413529 @default.
- W4287332019 hasConcept C121332964 @default.
- W4287332019 hasConcept C138885662 @default.
- W4287332019 hasConcept C150807984 @default.
- W4287332019 hasConcept C154945302 @default.
- W4287332019 hasConcept C18903297 @default.
- W4287332019 hasConcept C2777299769 @default.
- W4287332019 hasConcept C2780388253 @default.
- W4287332019 hasConcept C33923547 @default.
- W4287332019 hasConcept C41008148 @default.
- W4287332019 hasConcept C41608201 @default.
- W4287332019 hasConcept C41895202 @default.
- W4287332019 hasConcept C62520636 @default.
- W4287332019 hasConcept C84211073 @default.
- W4287332019 hasConcept C86803240 @default.
- W4287332019 hasConceptScore W4287332019C106301342 @default.
- W4287332019 hasConceptScore W4287332019C11413529 @default.
- W4287332019 hasConceptScore W4287332019C121332964 @default.
- W4287332019 hasConceptScore W4287332019C138885662 @default.
- W4287332019 hasConceptScore W4287332019C150807984 @default.
- W4287332019 hasConceptScore W4287332019C154945302 @default.
- W4287332019 hasConceptScore W4287332019C18903297 @default.
- W4287332019 hasConceptScore W4287332019C2777299769 @default.
- W4287332019 hasConceptScore W4287332019C2780388253 @default.
- W4287332019 hasConceptScore W4287332019C33923547 @default.
- W4287332019 hasConceptScore W4287332019C41008148 @default.
- W4287332019 hasConceptScore W4287332019C41608201 @default.
- W4287332019 hasConceptScore W4287332019C41895202 @default.
- W4287332019 hasConceptScore W4287332019C62520636 @default.
- W4287332019 hasConceptScore W4287332019C84211073 @default.
- W4287332019 hasConceptScore W4287332019C86803240 @default.
- W4287332019 hasLocation W42873320191 @default.
- W4287332019 hasOpenAccess W4287332019 @default.
- W4287332019 hasPrimaryLocation W42873320191 @default.
- W4287332019 hasRelatedWork W2008992764 @default.
- W4287332019 hasRelatedWork W2030128136 @default.
- W4287332019 hasRelatedWork W2037900003 @default.
- W4287332019 hasRelatedWork W2041391279 @default.
- W4287332019 hasRelatedWork W2315492480 @default.
- W4287332019 hasRelatedWork W2322267203 @default.
- W4287332019 hasRelatedWork W2774791677 @default.
- W4287332019 hasRelatedWork W3012966034 @default.
- W4287332019 hasRelatedWork W3080663663 @default.
- W4287332019 hasRelatedWork W4287332019 @default.
- W4287332019 isParatext "false" @default.
- W4287332019 isRetracted "false" @default.
- W4287332019 workType "article" @default.