Has anyone tried polars in place of pandas? I’m very curious about this. github.com/pola-rs/polars

Polars ❤️ sorted data

Sorted data has a lot of structure, so it would be great if a dataframe library took advantage of this.

polars data does just this with fast-track algorithms on sorted columns...
🧵

Curious about Polars?

Explore it in this upcoming workshop and see if it makes sense for your team.

store.metasnake.com/2022-11-08-pol…

Python polars 0.14.22 is released.

github.com/pola-rs/polars…

Have you heard of this Polars library? Let's dive in?store.metasnake.com/2022-11-08-pol…

Sunday thought: how is it possible that the Netherlands is such a massive open source powerhouse, being a comparatively small country? Lots of projects (old and new) I follow were created there. It's so inspiring ♥️

(Tweet triggered by Humane Tech Community - #JoinFediverse)

#Python polars 0.14.19 is released. This is the first of many releases where we will improve the capabilities to process larger than RAM data.

This release on that topic:

- first parts of streaming engine
- batched csv reader

github.com/pola-rs/polars…

📢We are delighted to announce our 2 speakers
- Using BERT to investigate cognitive biases in thinking of everyday unexpected events' by Molly Quinn
- 'Hello #Polars !' by Antonio Bevilacqua
🎟 meetup.com/pyladiesdublin…

#Rust #Python #HuggingFace #Data

It bugged me that I needed to give this answer to JD Long, so I started making polars data a hybrid streaming engine. In the coming months we will support more streaming queries.

Today I was able to run this query on a 80GB CSV, needing < 1GB of RAM on 12 threads.

Python polars 0.14.16 is released:

github.com/pola-rs/polars…

See tweet below why this can make your joins much faster!

A vertical concatenation won't work as it doesn't like schema changes.

This is where diagonal concatenation in polars data comes in.

Diagonal concatenation appends your new records and add nulls where the data is missing for the old stuff. Sorted.

7️⃣For example, what about analyzing more than 1 million Stack Overflow questions? With Orchest and polars data, it’s easy! orchest.io/blog/the-great…

Next release of polars will ship with automatic subplan elimination. Complicated self-referencing plans will likely run much faster if your threads were contending.

LazyFrame.cache() is an operation a user can start to forget.