ODSC: Analyzing Complex Networks Part 2

This is part two of a brief series sharing components of my presentation titled Analyzing Complex Networks Using Open Source Software at ODSC East in Boston on May 21st. The first post looked at a few examples from a Boston Red Sox players network, while this one examines a Miles Davis album and musician network. I’ll share a few examples of network analysis within the context of the Miles Davis graph.

The Miles Davis network could be described as a tripartite network, or one with three layers. Miles is at the center, and connects to each of nearly 50 recordings. Other musicians then connect to the respective recording(s) they played on, but not to one another. This approach provides a very clear look at musical phases in the career of the legendary trumpeter, without the graph being clouded by excessive detail. Here’s a view of the final network, after which we’ll look at some components of the graph.

miles_1

We see some interesting patterns in the graph, specifically in viewing the pink circles, which represent individual albums. Musicians playing on a recording can be seen adjacent to that recording, except in the case of musicians present on multiple albums. We would expect them to be positioned relative to all of the recordings they played on. A quick visual scan leads to five distinct clusters, as seen in the next screenshot.

miles_2

Now that we have identified these clusters, it would be helpful to understand their meaning and relevance to Miles career. Using the graph in interactive fashion, we can learn more about the recordings and musicians, and begin to formulate some insights. These can be confirmed by referring to album links on the web or in Wikipedia, which give context to what we are viewing. Based on these steps, here is a quick overview of the five clusters.

miles_3

A final step might be to add some verbiage using PowerPoint or Inkscape, which I’ve done below in very minimalist fashion. We could also add this to a web version using CSS attributes to position the text, although this could get tricky as we pan and zoom on the graph. We might be better off using some sort of stylized marker (color or shape) to communicate some of this information.

miles_4

There is much more that could be done, but I hope this brief example shed some light on the usefulness of network graphs, especially from a pure visual perspective.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

ODSC: Analyzing Complex Networks Using Open Source Software

I’ll be presenting at the 2016 ODSC East event in Boston May 20-22. ODSC stands for Open Data Science Conference, where the focus is on using open data or open source tools to do clever things in the information space. The topic of my presentation is Analyzing Complex Networks Using Open Source Software, where I’ll talk through several example networks built using Gephi and Sigma.js.

While the slides are not all prepared at this stage, I’ll share a few bits that will wind up in the talk. My goal is to convey to the audience how networks can be used to statistically and visually understand complex information. After providing an overview of network analysis (at a very high level), I’ll be sharing slides from three very different networks – a Miles Davis album network (created in 2014 and rebuilt in 2016), a Boston Red Sox player network (also built in 2014), and a brand new example using data from the amazing GDELT Project.

Here’s a glimpse into what I’ll be sharing, starting with the Red Sox examples, where we examine the networks of three well known players from the last 100 years. First, Ted Williams network:

odsc_williams

Followed by Carl Yastrzemski:

odsc_yaz

Now Jason Varitek, longtime catcher and captain for two World Series championship teams:

odsc_varitek

In talking through each of these networks, I will attempt to highlight some differences in their respective structures based on the era in which each player spent time with the Red Sox. For example, there are many more connections in the Varitek network compared to Williams and Yaz, despite a shorter duration with the team. Why would this be the case? Perhaps spending time in the era of higher salaries, larger pitching staffs, and the evolution of free agency might go a long way towards explaining why Jason Varitek crossed paths with far more players than did his earlier predecessors.

Stay tuned for additional posts featuring the Miles Davis and GDELT networks.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

Warsaw Data+ Presentation

I have the good fortune to be the keynote speaker at this year’s Data+ conference in Warsaw, Poland on November 26th, so the traditional American Thanksgiving meal will not be in store for 2015. This is a very exciting opportunity, and comes on the heels of having presented in Boston at the Data Visualization Summit in September 2015, so it’s been a busy last few months getting presentations squared away.

My topic at the conference is Data Driven Storytelling, where I’ll walk the audience through some of my approach and philosophy about using data visualization to deliver information and insights about specific topics. In addition to the talk, I’ve created a story on my visualidity.com site that chronicles the last 21 seasons of play in the Ekstraklasa, the top level of play in Polish football.

Thus far it has been an absolute joy working with the folks at IDG/Computerworld, who are responsible for running the event. Patrycja Kuriata, Program Director for the conference, has been incredibly responsive and helpful with any questions or details, and has made the entire process a pleasurable one.

I’m putting the wraps on my content as October comes to an end, and look forward to visiting Poland in a few weeks, and reporting back on the conference as well as on the few days of sightseeing in my plans.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

Eyeo Day 4 – Reality, Fantasy, and Being Human

Day 4 (the final day) at Eyeo is always a little bittersweet, knowing that the great energy one feeds off is about to come to an end. At the same time, there is still one more day of great talks, followed by a closing party that provides yet another opportunity to talk with creative types from a variety of places and backgrounds.

Leading off the day’s schedule was Nicky Case, heretofore unknown to me and perhaps many others in the room. That was about to change, as Case took us on a tour of his personal and professional life, delivered with great panache. Turns out he is a masterful storyteller, embodied by his interactive work on projects like the ‘Parable of Polygons’ and ‘Explorable Emotions’. Every year, there are at least a couple talks that go way beyond expectations, and this turned out to be one of them for this version of Eyeo.

Next up was Beatrice Lartigue, a French artist who walked through a few of her interesting projects, including some interactive installations. Of particular note was an active learning project based on Prokofiev’s classic ‘Peter and the Wolf’.

The afternoon began with ‘Mapping Police Violence’ presented by Deray McKesson and Sam Sinyangwe. McKesson has employed Twitter as a powerful platform for protest, while Sinyangwe has taken the route of documenting police violence by mapping incidents, thus allowing for a more factual approach to identifying police forces with chronic issues.

To cap the afternoon session, Nick Hardeman and Theo Watson of design i/o took the audience through a magical tour of their ‘Connected Worlds’ project, now installed at the New York Hall of Science. This highly interactive exhibit is full of engaging characters that will surely draw children in while simultaneously teaching lessons about interactions with nature.

On to dinner, once again navigating my way to the hot North Loop area, this time to Borough. Borough not only has some terrific food, but also offers a very intriguing wine list with seldom seen options from across the globe. A couple glasses of wine, a delightful halibut terrine, and a very good guinea hen dish later, it was time to head to Nicollet Island for the closing talks and party.

Eyeo veterans Jake Barton and Zach Lieberman closed this year’s festival, with Barton discoursing on memory and future, as seen through a compelling exhibition created for the 9/11 Museum. Lieberman delivered a heartfelt, emotional farewell to his recently departed father, who told him that ‘storytelling is not about technique, but being fully human.’

The final party provided an opportunity to chat with a few more festival friends, prior to becoming a bit melancholy when the realization sinks in that Eyeo is coming to a close for another year. I took one last look at the creative people talking, sharing, and enjoying the scene, before electing to walk back to the hotel. A long walk seemed to be the best way to process my thoughts, think about what I learned and who I met, and how it might inform and inspire my work over the coming months. So long, Eyeo.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

Eyeo Day 3 – Words, Maps, Code

Day 3 at Eyeo had another intriguing assortment of speakers and topics to learn from, as well as the final planned night at Nye’s prior to its August closing. Starting the day in the McGuire Theater was Allison Parrish, who delivered an intriguing talk on words and semantic tools. This is an area of interest for me, specifically in striving to visualize word connections and context, so there was much to like about the talk. Parrish has put together several interesting Twitterbots, including the Power Vocab Tweet, Library of Emoji, and Deep Question Bot, all of which I am adding to my following list. Often silly yet clever use of semantics in the Twitter space.

Next up was Ingrid Burrington, who posed some challenging questions about the scope and invasiveness of fiber optic paths and other high tech connectivity infrastructure.

The afternoon session began with one of my favorite presentations of the festival, delivered by Ben Vershbow and Mauricio Giraldo from the NYPL Labs team. NYPL stands for New York Public Library, filled with incredible resources that Vershbow and Giraldo shared. Funny, engaging, and informative, attendees were taken through some of the great work going on at the library, including the oldnyc.org project which maps historical photos, and the community sourcing of Building Inspector, with it’s classic motto – Kill Time. Make History. Great work, fantastic talk.

Next up, Harlo Holmes talked about a few of her projects and interests, with a focus on security tools that protect users from surveillance and intrusion.

To finish the day, Ramsey Nasser presented a splendid talk on coding that was so much more than that. He talked about the need to expand the world of coding to encompass more than the traditional English-only, left to right text that underpins virtually all coding frameworks and languages. Nasser was very entertaining while driving home an important message about the need to make revolutionary changes if we are to maximize the potential for coding.

Another great set of daytime talks, and now there was to be some downtime before heading to dinner and the evening gathering at Nye’s. At least that was the original plan, until the great folks at M|I|C/A (Maryland Institute College of Art) announced a happy hour at The Third Bird, just across Loring Park. Not wanting to disappoint a generous sponsor, I joined dozens of others for a couple beers before making the 1.5 mile walk back to the hotel in a steady rain. One must be able to make sacrifices!

Dinner was planned for Pizza Nea, a spot I visited during the 2012 Eyeo Festival, for one of their excellent thin crust pies. Expecting a slim crowd on a rainy Wednesday at 8:00, I was surprised to find a nearly full restaurant. Taking a seat at the bar adjacent to the pizza making area, I was informed that I just missed perhaps their busiest Wednesday in memory. Why? Unbeknownst to me, the Rolling Stones were playing at the nearby University of Minnesota football stadium that evening, which presumably filled all the restaurants in the neighborhood prior to the show.

Finally, I made the two block walk to Nye’s, joining dozens of other Eyeoans for beer, booze, and piano bar frivolity, with Jer Thorpe in particularly good voice for his annual rendition of Neil Diamond’s Sweet Caroline. Where else but Eyeo!

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

Eyeo 2015 Day 1 – Dear Data

The Eyeo crew changed things up a bit this year, using the Guthrie Theater as the opening night venue after a few years at Aria, which was bumped to Tuesday night (Day 2) this time around. Moving festivities to the Guthrie gave attendees a glimpse at this major venue and it’s footprint on the Mississippi River, while also taking Eyeo to the Mill District, another of Minneapolis reborn neighborhoods. Personally, it provided an opportunity to stroll a few new streets and put even more mileage on my already tired legs and feet.

Part of the beauty of Eyeo is the ability to chat with other attendees from around the globe who descend on Minneapolis each June, eager to learn and share. After exchanging pleasantries with a few folks, I had the good fortune to engage in a long conversation with Roman Verostko, one of the pioneers of algorithmic art, and a presenter at Eyeo 2014. Roman is a couple generations older than the typical Eyeo attendee, but retains a curiosity that I find both admirable and enviable. After talking about his art, Minneapolis evolution, Detroit (my town), and a host of other topics, we descended into the theater to learn more about Dear Data, a collaborative project from Eyeo regulars Giorgia Lupi and Stefanie Posavec.

The premise behind Dear Data is fascinating – a weekly exchange of postcards on a single topic. Given that both Lupi and Posavec are exceptional illustrators, the postcards were visually engaging, informative, and often downright hilarious. The topics included ‘a week of physical contacts’, ‘a week of complaints’, ‘a week of drinks’, and so on. The desire to learn more about one another served as the fuel for the project; the pair first met at Eyeo 2013, but live on different continents, and wanted to find a way to learn more about each other through this project. The presentation was fun and engaging while offering a window into their respective quirks, habits, and drawing styles.

As mentioned in a prior post, I love to eat and drink when I come to Eyeo, and this year is proving to be no exception. After meeting a friend for lunch at Dan Kelly’s (very good chicken pot pie), I later visited Freehouse, a brewpub in the hot North Loop district to sample some of their wares. From there it was a short walk to Black Sheep Pizza to grab one of my favorite pies on the planet – the olive, sausage, salami, and onion version, with beautifully charred edges courtesy of the coal-fired oven. Then on to the Guthrie, a nice little 1.2 mile walk to burn off some of the recently acquired calories.

All this activity had me too tired to stay for the entertaining Ignite sessions which followed the Dear Data talk, so I’ve had to hear about those second-hand. Feedback has been positive, especially for the ‘mayonnaise’ talk. I’ll have to wait for the video to see what everyone’s talking about. Next post will recap Day 2 – see you soon.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

2015 Eyeo Itinerary

I’m back at the Eyeo Festival for the 4th consecutive year, hoping to absorb some of the creative energy that takes place every June in lovely Minneapolis. As usual, there are a handful of folks who will be sharing their insights in ways that can help me to perfect my craft, but I also look forward to a surprise or two that forces me to re-imagine possibilities. In addition to the great events taking place within the Eyeo sphere, I always look forward to a bit of dining and drinking while here, partaking of the increasingly exciting restaurant and brewery scenes in MSP, and will put in a few words about the establishments visited while here. Finally, each day I plan to add a post recapping the previous day’s events and my take on what it all means.

Among the folks I’m looking forward to hearing from:

  • Giorgia Lupi and Stefanie Posavec as opening night keynote speakers. Each of these talented ladies are Eyeo regulars who possess fantastic artistic abilities that trigger my imagination. While I could never hope to duplicate their respective artistry, their visual ideas inspire me to think about the many ways data can be displayed.
  • Ben Fry is back after a year’s absence, and will no doubt share some impressive work done at his Fathom studio
  • Amanda Cox of the New York Times visualization team is back, and will have some outstanding work to show

There are also a few newcomers I’m eager to see – Mario Klingemann, Shazna Nessa, and Ingrid Burrington among them.

Now for the food aspect. After arriving 3+ hours behind schedule, I still managed to make the 1.2 mile walk from the hotel to the trendy North Loop area for a bite or two at Spoon and Stable, one of the hottest venues in town. Fortunately, there is available bar seating with access to both the bar and full menus, so walk-ins can be accommodated, especially late on a Sunday evening. Created inside a former stable building, the restaurant has a great vibe, friendly staff, and a creative menu that includes items ranging from duck meatloaf sliders through Colorado lamb. I consulted with one of the bartenders and wound up having bison tartare followed by spaghetti nero (squid ink pasta with octopus, mussels, and prawns), accompanied by a bit of red wine (there is a nice selection of local beer available as well). Both dishes were flavorful, creative, and delicious, and the overall experience was very favorable.

Tonight is likely to see a return visit (I’ve been the previous 3 years) to Black Sheep Pizza for some of their outstanding coal-fired pie, before taking the walk down to the Guthrie Theater for opening night. More to come tomorrow.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

Day 3 Eyeo Festival Recap

Well, it’s come and gone again, like a fleeting annual romance, a la Same Time Next Year, the Alan Alda & Ellen Burstyn feature about a couple that meets for a single weekend once a year. This is how I feel about the Eyeo Festival, where once a year for the last three years I have this short flirtation with this incredible event, meeting new people, reacquainting with others, and opening my mind to new ideas and people. So it was on the final day of this year’s event, with more amazing speakers sharing the work they create.

Morning began with Lauren McCarthy, recognizable from prior festivals, but presenting for the first time at Eyeo. She shared a creatively diverse group of projects she has worked on dating back a number of years, some merely playful, others representing bold forays into uncharted territory, many revolving around human relationships and interaction. One of the projects even netted her a bit of controversy, including some discussion on Fox news, hilariously shared with the Eyeo audience. Here are some of her gems: Crowdpilot, Us+, and Inneract.

Next up was Eric Rodenbeck, head of the great Stamen map and visualization design studio. Rodenbeck shared a number of Stamen projects, and also provided a great deal of insight on how to keep a small design studio going for better than a dozen years. Stamen does a lot of impressive work, and shares much of it in the public domain, including their Map Stack application. Lots of great advice provided for those out on their own, but also much relevance to tinkerers like myself working in large organizations.

One of the lunchtime sessions featured Ben Jones of Tableau, sharing some of the capabilities of the new Tableau 8.2, to be released very soon. I’m already very familiar with Tableau through daily work use, so much of the material was not new, but some of the upcoming 8.2 features are exciting indeed. After the talk, I had the opportunity to speak with both Ben and the Tableau mapping lead. Maps in 8.2 were designed in conjunction with the aforementioned Stamen, so they are likely to be a significant improvement over the already respectable maps in prior Tableau versions.

After lunch, my first stop was to hear the irreverent Jessica Hagy, who has done more to make the index card useful than anyone in recent memory. Her simple line drawings, typically either x-y axes or Venn diagrams, merge often unlikely pairs of ideas, items, or things, and hilariously plot them. Hagy has a number of other interesting projects, so take a moment to check them out.

Roman Verostko won honors as the oldest speaker (and attendee) at this years festival, and has been creating art projects since before most attendees were even born (before many of their parents were born, even!). At 85 years young, Verostko was the creator of a code-based machine to control brush strokes, in a process known as algorithmic painting. He also spents many years as a Benedictine monk, before leaving the monastery to become a full-time artist and professor. Versotko was one of the three pioneers featured at Eyeo, along with Frieder Nake and Lillian Schwartz.

The festival ended for me with the great Santiago Ortiz, creator of some of the most amazing interactive projects to be found on the web in recent years. Ortiz, who is based in Argentina, took the audience through a talk primarily focused on a 6-month period characterized as the most stressful, challenging, and uncertain period of his life, after parting ways with a previous employer and setting out on his own. After a number of physical hardships related to stress and fatigue, he eventually triumphed and began creating some incredible work that has to be seen by all visualizers. Ortiz work is heavily influenced by his background in and love for mathematics, which results in some incredibly elegant work. This was a very inspiring end to my third Eyeo trip.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

Eyeo Festival Day 2 Recap

Writing this from the charming confines of the Bull Run coffee shop on Lynvale, nestled in a mostly residential part of South Minneapolis, and reflecting on another great day at the 2014 Eyeo Festival over a real deal croissant and some great coffee. It’s a fantastically clear, cool morning in Minneapolis, and I have a spot in front of an open window. Life is good.

So what were the highlights from another day at Eyeo? Let’s take a chronological look this time, beginning with Kim Rees discussion on the future of data. One of the hot topics this year, unsurprisingly, is the entire Snowden/NSA/surveillance/privacy discussion, and how best to manage this entire spectrum in the future. Rees delivered some interesting scenarios, potentially controversial viewpoints, and some tantalizing glimpses into the future of data, included the highly overused ‘Big Data’ term, soon to be replaced by ‘f…ing humongous data’. A major takeaway from her session was that data needs to be widely available, and we somehow have to manage it so that good triumphs over evil (NSA & friends).

Next up for me was Eric Fischer, a mapping and geo data specialist, who is doing some fascinating work with geographic data. One of his focuses (obsessions?) revolves around understanding what makes urban areas walkable. His work was a fascinating litany of examining pedestrian patterns using a variety of sources, including Census data, Twitter tweets, and government data with actual volume counts at intersections in various cities. Great stuff (full disclosure: I love mapping), and I even had the opportunity to spend five minutes speaking with him later in the day about mapping and urban solutions.

After lunch in the sculpture garden, it was back to work taking in a panel discussion on Augmented Education, where the discussion was around providing tools that can help the education community and students to embrace creativity inside and outside the classroom. Great thoughts from all panelists, including Martin Wattenberg and Fernada Viegas, Amit Pitaru, Evelyn Eastmond, and Jer Thorp. Among the interesting solutions discussed was an application for kids to share their creative work known as Scratch.

My day concluded with the creative Darius Kazemi, a web designer by trade who does fantastic (and frequent) small projects on the side; 73 of them in 2013 alone! He took us through a tour of some of his creative, and often humorous work, including Two Headlines, where trending Twitter topics are randomly merged with one another to form often hilarious combinations. Kazemi is also well known for his Random Shopper app, which buys random items on Amazon (limited to books and CDs) each month, and ships them to Kazemi’s residence. His work perfectly embodies what I love about Eyeo – someone previously unknown to me who is doing great, creative work that broadens my thinking.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather

More Eyeo Festival Mind Expansion

As expected, the first full day at the 2014 Eyeo Festival was mind blowing, and what was interesting is that some of it came from unanticipated sources. While I knew that Mike Bostock, Nicholas Felton, et al would provide interesting and oftentimes amazing content, a few other presentations were what really blew me away.

Wrapping up my day (and being a bit fatigued in the process), was Brian House, holding court in the McGuire Theater. House is a multimedia artist who does some fascinating explorations using sound and data, and has even written musical pieces based on unlikely data sources, such as the black box information from the highly publicized vehicle crash of Massachusetts Lt. Governor Tim Murray. Several really interesting projects were shared, and have now got me thinking about the intersection of data and sound, and how it could be used in my work, albeit in a slightly different vein from what House does. Great stuff.

The evening sessions were back at Aria, and featured Cesar Hidalgo and Kate Crawford, two of the top people in their fields, and with two very contrasting, but equally effective presentation styles. Hidalgo was rapid fire, excitable, and intense while presenting some great explorations on why some countries (cultures?) have been successful in becoming more prosperous, while others have languished. At its root, the thesis proposed that know-how is more important than knowledge, especially the ability to learn how to produce economically valuable goods. He used the case of Peru versus South Korea to make his point convincingly. Remarkably, Peru had nearly double the per capita wealth of South Korea in the 1960s, but has made no further advancement, while South Korea now has 4 times the per capita income of Peru. Why? Because South Korea had the know how to build goods such as vehicles, jets, consumer electronics, etc. while Peru continues to grow fruit. Fascinating analysis, delivered in much greater detail than I can provide here.

Crawford, on the other hand, is polished and charming, with an Australian accent that has just the right amount of Down Under lilt for American ears. Her exploration of Snowden, the NSA, security, and big data was outstanding and insightful, and posed a number of valuable questions about what the future holds. A bit of comedy was present as well, via the amateurish, almost child-like slides created by NSA functionaries. These guys can’t even make a decent Powerpoint slide, and they use cheesy clip art to boot. Stuck in the 1990s, like so many sectors of government, yet still with the potential to ruin vast amounts of lives, courtesy of the explosion in available data.

Just three examples of why I love this event, and I haven’t even touched on the human aspect; this is a group that cares about the future, and is determined to have a say in it, rather than trusting big institutions to do the job. Very refreshing for my somewhat older ears to hear the passion at Eyeo. More to come.

FacebooktwitterredditpinterestlinkedinmailFacebooktwitterredditpinterestlinkedinmailby feather
FacebooktwitterlinkedinrssFacebooktwitterlinkedinrssby feather