Convert a Spark dataframe into a JSON string, row by row

Not all schemas are created equal. Sometimes, no matter how much you massage the structure, you want to make sure and future-proof your work. That’s what this is all about. Taking the original data from a dataframe, and making a JSON representation of it in a single column. That way you can be sure and maintain all of your data long term. …

this post is featured

Airflow's Strengths and Weaknessess, and Dynamic Tasks

I had the opportunity to speak at the New York Big Data Warehousing meetup. We had a great attendance with over 200 registrations. I co-spoke with the team at SoulCycle, and what a great group they are. In my presentation I talk about what Airflow is, and what it isn’t. Some ways to avoid typical pain points, and the real power of dynamic tasks. You can grab the whole slide deck below or watch the presentation here. It was a blast and a great audience. …

WordCamp SLC 2015

WordCamp Salt Lake City 2015 is great. Here I am minutes from presenting, so I thought I’d share my slidedeck and GitHub project I made specifically for my talk. I’ll link the video when it’s fully uploaded. For the basis of this discussion I dug really deep into internationalization, and how to avoid some of the pains that some of us face. Again, I’ll add more here. For now, here are the resources you may want to take a look at. Slide Deck      GitHub Repo …

RecordSeek, Redesigned From the Ground Up

RecordSeek, for those who don’t know, is a powerful utility to attach webpages as sources to FamilySearch.org. Two weeks ago our service went down. Why did it go down? Well, when I created RecordSeek the FamilySearch Platform API (the way third parties connect to FamilySearch) was brand new. In fact, I was the first one to use it. But, I didn’t exactly use it the proper way (that was still being created). I connected through the FamilySearch site code, which was a proxy to the Platform API. Long story short, FamilySearch changed their code and RecordSeek went down. …

Suggestions for the Theme Review Team

My name is Dovy Paukstys. I am the co-founder and lead developer of Redux Framework. We represent over 7500 known developers with many of products, which serve close to 1+ million users. Our developers are from all across world, many of which speak different languages. I am lesser known in the WordPress community directly, coming onto the scene in 2013 and not spending much time on the WordCamp circuit. During my time in this industry, I have grown Redux Framework from 200 developers, merged with 3 other frameworks, consequently creating the most powerful WordPress options framework to date. …

Article on Tuts+

Not much to post here, just an announcement. I was able to publish my first article on Tuts+ today. I’ve been in the WordPress scene since 2007, and I have never tried to get published. So when a situation presented itself, I decided to go for it. It’s pretty basic information, but I think it’s really valuable. …