1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
1️⃣Byte Latent Transformer: Patches Scale Better Than Tokens
what: train llama on raw bytes without a fixed vocabulary. - dynamically patches bytes usign local small encoder - main decoder process these patch in AR setting - local deocder makes next byte prediction. paper:https://arxiv.org/abs/2412.09871
2️⃣Large Concept Models: Language Modeling in a Sentence Representation Space
what: work with entire sentences as "concepts" through SONAR embeddings. - quite similar with the first paper here, but it merges tokens into high dim embeddings - working with sentence-level embeddings directly.
3️⃣GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy
what: Created a diffusion model for probabilistic weather forecasting that generates 15-day predictions with 12-hour steps how: - It aggregates two previous timesteps to predict the next weather state - Instead of directly sampling weather state, it generates residuals (differences) relative to the previous state. - Артемий в канале AI для Всех сделал ревью на русском, почитайте.
my thoughts: Looks like we're finally getting closer to how humans actually process language, not just crunching tokens like robots. Whether it's patching bytes or bundling tokens into sentence embeddings, this hierarchical approach seems to be the way forward. GenCast - is just super interesting adoption of modern AI to real problems in natural science.
BY the last neural cell
Warning: Undefined variable $i in /var/www/group-telegram/post.php on line 260
As a result, the pandemic saw many newcomers to Telegram, including prominent anti-vaccine activists who used the app's hands-off approach to share false information on shots, a study from the Institute for Strategic Dialogue shows. In a statement, the regulator said the search and seizure operation was carried out against seven individuals and one corporate entity at multiple locations in Ahmedabad and Bhavnagar in Gujarat, Neemuch in Madhya Pradesh, Delhi, and Mumbai. That hurt tech stocks. For the past few weeks, the 10-year yield has traded between 1.72% and 2%, as traders moved into the bond for safety when Russia headlines were ugly—and out of it when headlines improved. Now, the yield is touching its pandemic-era high. If the yield breaks above that level, that could signal that it’s on a sustainable path higher. Higher long-dated bond yields make future profits less valuable—and many tech companies are valued on the basis of profits forecast for many years in the future. "There is a significant risk of insider threat or hacking of Telegram systems that could expose all of these chats to the Russian government," said Eva Galperin with the Electronic Frontier Foundation, which has called for Telegram to improve its privacy practices. In the United States, Telegram's lower public profile has helped it mostly avoid high level scrutiny from Congress, but it has not gone unnoticed.
from us
