1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
BY the last neural cell
Warning: Undefined variable $i in /var/www/group-telegram/post.php on line 260
Such instructions could actually endanger people — citizens receive air strike warnings via smartphone alerts. If you initiate a Secret Chat, however, then these communications are end-to-end encrypted and are tied to the device you are using. That means it’s less convenient to access them across multiple platforms, but you are at far less risk of snooping. Back in the day, Secret Chats received some praise from the EFF, but the fact that its standard system isn’t as secure earned it some criticism. If you’re looking for something that is considered more reliable by privacy advocates, then Signal is the EFF’s preferred platform, although that too is not without some caveats. The Dow Jones Industrial Average fell 230 points, or 0.7%. Meanwhile, the S&P 500 and the Nasdaq Composite dropped 1.3% and 2.2%, respectively. All three indexes began the day with gains before selling off. Oh no. There’s a certain degree of myth-making around what exactly went on, so take everything that follows lightly. Telegram was originally launched as a side project by the Durov brothers, with Nikolai handling the coding and Pavel as CEO, while both were at VK. The SC urges the public to refer to the SC’s I nvestor Alert List before investing. The list contains details of unauthorised websites, investment products, companies and individuals. Members of the public who suspect that they have been approached by unauthorised firms or individuals offering schemes that promise unrealistic returns
from us
