1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
BY the last neural cell
Warning: Undefined variable $i in /var/www/group-telegram/post.php on line 260
At the start of 2018, the company attempted to launch an Initial Coin Offering (ICO) which would enable it to enable payments (and earn the cash that comes from doing so). The initial signals were promising, especially given Telegram’s user base is already fairly crypto-savvy. It raised an initial tranche of cash – worth more than a billion dollars – to help develop the coin before opening sales to the public. Unfortunately, third-party sales of coins bought in those initial fundraising rounds raised the ire of the SEC, which brought the hammer down on the whole operation. In 2020, officials ordered Telegram to pay a fine of $18.5 million and hand back much of the cash that it had raised. False news often spreads via public groups, or chats, with potentially fatal effects. Ukrainian forces have since put up a strong resistance to the Russian troops amid the war that has left hundreds of Ukrainian civilians, including children, dead, according to the United Nations. Ukrainian and international officials have accused Russia of targeting civilian populations with shelling and bombardments. Although some channels have been removed, the curation process is considered opaque and insufficient by analysts. Andrey, a Russian entrepreneur living in Brazil who, fearing retaliation, asked that NPR not use his last name, said Telegram has become one of the few places Russians can access independent news about the war.
from id
