1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
BY the last neural cell
Warning: Undefined variable $i in /var/www/group-telegram/post.php on line 260
"There are several million Russians who can lift their head up from propaganda and try to look for other sources, and I'd say that most look for it on Telegram," he said. I want a secure messaging app, should I use Telegram? Markets continued to grapple with the economic and corporate earnings implications relating to the Russia-Ukraine conflict. “We have a ton of uncertainty right now,” said Stephanie Link, chief investment strategist and portfolio manager at Hightower Advisors. “We’re dealing with a war, we’re dealing with inflation. We don’t know what it means to earnings.” In a statement, the regulator said the search and seizure operation was carried out against seven individuals and one corporate entity at multiple locations in Ahmedabad and Bhavnagar in Gujarat, Neemuch in Madhya Pradesh, Delhi, and Mumbai. "We're seeing really dramatic moves, and it's all really tied to Ukraine right now, and in a secondary way, in terms of interest rates," Octavio Marenzi, CEO of Opimas, told Yahoo Finance Live on Thursday. "This war in Ukraine is going to give the Fed the ammunition, the cover that it needs, to not raise interest rates too quickly. And I think Jay Powell is a very tepid sort of inflation fighter and he's not going to do as much as he needs to do to get that under control. And this seems like an excuse to kick the can further down the road still and not do too much too soon."
from ua
