The Value I Gleaned from the First Daugherty Hackathon

When I learned Daugherty was having an internal hackathon in St. Louis, and was offering a $10,000 prize for the team that produced the best solution, I flew from Minneapolis with a few of my peers.

We arrived about 4 p.m. After a warm welcome, we received information about the project we’d build. The subject of the hackathon was data tracking and privacy and security. The use cases included:

  • Store and restrict information collected about the user to the user’s country.
  • Allow the user to completely erase themselves and any data gathered on them from our system.
  • Require consent and some age (based on country) from the user prior to information tracking.
  • Allowing anyone (not just users) to complain to the “commissioner” without an account.
  • Informing users about any information sharing done.
  • Informing users based on what country they’re in what and why we’re tracking them.

We were inspired to meet these requirements by providing a credit bureau system, and we had 40 hours until the presentation after getting the requirements. Little sleep was had, much cold-press coffee (delicious by the way) was drunk. By the end of our time, we did have a product, but it wasn’t as functional as we’d hoped it would be.

Team Daugherty Hackathon 2017 from Daugherty Business Solutions on Vimeo.

The Three Lessons We Learned through the Hackathon Experience

1. When you have to work fast (which is often), be agile. We followed this philosophy pretty closely, but not closely enough. After getting the requirements, we went to the whiteboard. We read through the use cases, created the workflows, then defined the data model. After that, we all got to work.

So where did we go wrong?

Pretty much right at the beginning. We defined the workflows, then defined the data model. That’s not the agile way. We were operating under the assumption that minimal viable product (MVP) was meeting all the bullet points outlined above, which turned out to be unrealistic for what we were making.

We got to the demo and had two to three workflows that were actually functional, and about six to seven that were about 60 percent complete. We’ve all been in “What you would see here if it were working” demos, and we went through this one due to lack of agility.

In hindsight, we should have done things more piecemeal. The story would change from this:

“The user logs in, clicks the ‘check credit’ button, enters their name, DOB, National Credit Identifier (NCI), address, etc. … then sees their credit score.”

to this:

“The user logs in, clicks check credit and sees their credit score.”

…upon completion…

“The user logs in, clicks check credit, verifies their name and sees their score.”

…upon completion…

Eventually, it would go back to the initial story. Instead, we ended up with about 10 required fields that once correctly entered, didn’t provide a credit score.

2. Code for your demo (as long as that’s all that matters). It might not seem like it, but eight minutes is not a lot of time. We ended up with a 30-minute application and eight minutes to show it. In retrospect, we should have coded for the demo.

Arguably, coding toward the demo in a standard engagement can sometimes be a mistake. However, it would not have been in this environment. Instead of spending 30-plus seconds (6.25 percent of our time) filling out 10 validation fields, we should have made the only field that matters your name.

We spent too much time doing “nice-to-have” features that the app would work without, rather than going back to the use cases and working on the easy-to-demo, low-hanging fruit.

3. Screenshots are your friend. We thought a live demo would be cool, but, realistically, given the time constraints (and thus fragility of the code) and the amount of functionality we were working in, a slideshow would have served us far better. We saw many other teams doing that, which was great.

Despite our failings, the Daugherty hackathon was a lot of fun. Everyone at the St. Louis office was very friendly and welcoming, and sleep-deprived people can do some pretty funny things (I spent a good 30 minutes figuring out why the app couldn’t find the “consumber” table — facepalm). Our project didn’t work out the way we wanted, but it was an incredible opportunity and learning experience. I am thankful to work for an organization that invests in its employees this way, and hope to participate in hackathons in the future.

JohnMollberg

John Mollberg started at Daugherty in June of 2016 after graduating from the University of Minnesota with a B. S. in Computer Science. He's been interested in technology and problem solving since a young age and still finds himself tinkering with new gadgets on a regular basis. He spends his free-time boxing, playing board and video-games, and going to concerts.

More than Just Design: How You Can Significantly Boost Performance with Code Optimizations

When it comes to technology, we are not particularly patient. A snappy user experience beats a glamorous one. We engage more with a site when we can move freely and focus on the content, instead of an endless wait. User Experience research recommends the following responsiveness guidelines:

  • Delays <0.1 seconds feel instantaneous.
  • Delays < 1 second are the limit for a user’s flow of thought to stay uninterrupted. The user will notice the delay but still be able to focus on the task at hand.
  • Delays between 2 to 10 seconds should have a spinning icon to visually indicate something is happening.
  • Delays > 10 seconds should have a percent-done indicator along with a cancel button.

Just to put these requirements in context, it takes between 0.1 and 0.4 seconds to blink. It takes six seconds to breathe. With these kinds of demands, if you don’t design with performance in mind, nobody will use the product. But performance isn’t as simple as you might think.

Why Design Is Critical in a World Where Not All Operations Are Created Equal

To deliver the functionality of an application, many different types of tasks need to be performed: reading data from a database, performing calculations, security checks and rendering User Interface (UI) components. From the perspective of performance, not all operations are created equal. Operations performed in memory are orders-of-magnitude faster than operations that require calls over the network.

For example, it takes 20 milliseconds to read 1 MB sequentially from a disk, whereas it only takes 250 microseconds to read 1 MB sequentially from memory.

When starting the design for your application, you will want to consider the most critical parts and come up with an architecture that is appropriate for the problem at hand. You will need to start with objective goals.

  1. How long should an average request take?
  2. How many concurrent users should the application support?
  3. What is the peak load the application must handle?
  4. What is the deployment environment for the application?

Think of it this way: If you’re using a hacksaw to clear-cut a forest, you are going to have a bad day. If you don’t have the right tools for the job, no amount of heroics is going to help.

Using a layered design will make identifying and correcting performance problems much simpler. There are many potential reasons for a sluggish application. Having multiple types of operations occurring within the same chunk of code can obfuscate which chunks are slow and why. It also makes it more difficult to optimize without breaking unrelated code.

How to Use Code Optimizations Effectively to Maximize Performance

Code optimizations change already-correct code so that it executes faster. To do it effectively, first define your optimization goals clearly and measure the impact of the changes you make using a profiling tool. Do not start performing optimizations without a profiling tool of some sort. Performance problems are often not obvious, and the changes you introduce to the code base could very well make it perform worse!

Before you optimize, you’ll want to make sure the code works. Then you’ll want to refactor it to make it right and easy to understand. Only then should you optimize it.

Code optimization looks different depending on the type of operation.

  1. For I/O-bound operations, you will want to first identify the costly operations during application profiling. This might include disk reads, web service calls or database calls. Once identified, you might consider rewriting the slow parts using either multithreading or batching:
    • Multithreading executes multiple threads simultaneously. If you multi-thread code, break the task into individual pieces that don’t require communication and locking. Race conditions and deadlocks are very difficult to identify and fix.
    • Batching updates the interface to perform a batch of work instead of making multiple network calls. By batching work, you limit the number of expensive network calls that are made.
  2. For CPU-bound optimizations, know your algorithms and data structures. Restructure your applications to use the most appropriate structure. For example, if you need to determine the intersection of two lists in memory, a hashset will be much faster than a list. The complexity class for performing a lookup is O(1) for a hashset instead of O(N) for a list. Comparing two lists with 100,000 items, that is a difference between 100,000 and 10 billion operations.

If you want to produce a product people will use, performance must be at the front of your mind. Ultimately, if the application design is faulty, code optimizations probably couldn’t save the application at the last minute. But once the design is right, code optimizations can be used to identify and remove the hot path within the application to really set your application apart.

At Daugherty, our Dev Center contains software engineers who have been involved in all aspects of delivering high-quality software: from requirements analysis to deployment automation. If you’re a developer ready to work for many different brand-name clients, developing and delivering pragmatic solutions to their business needs, join our team today.

JasonTuran

Jason Turan is a Senior Consultant at Daugherty Business Solutions. He has worked on a variety of platforms and industries since 2008 and particularly enjoys creating simple yet flexible applications.

Privacy by Design: Why It’s Critical to Your Organization

No surprise: Privacy is a big deal.

If your organization is storing data about people, privacy should be a big deal to you. This is especially true if you’re storing data about people in the European Union (EU), in light of General Data Protection Regulation (GDPR), which will go in effect May 25, 2018.

GDPR is designed to strengthen and unify data protection for all individuals within the EU and give control back to citizens over their personal data. Under GDPR, organizations in breach can be fined up to four percent of their annual global turnover (revenue) or €20 million (whichever is greater).

This is the maximum fine, which can be imposed for the most serious infringements, like not having sufficient customer consent to process data.

And it’s per infringement.

So what precisely does it mean to have data that aren’t properly anonymized?

The Look of Data that Can Incur Hefty Fines

Weak anonymization algorithms are one way of violating user privacy.

Remember Where’s Waldo? In the books, Waldo is hidden among a large crowd, and we are invited to pore over the pages, scanning for his trademark red-and-white-striped shirt, bobble hat and glasses. Knowing what to look for makes it slightly easier, although the books introduce red herrings to make Waldo more difficult to spot.

Imagine if an entire Where’s Waldo? illustration contained a mass of people all wearing dull green, surrounded by dull green landmarks, and Wally in his trademark red.

He’d be really easy to spot.

A GDPR infringement occurs if somebody can determine a person’s identity through data, even if anonymization algorithms are in place. Best intentions don’t matter.

One of the best methodologies your organization can institute to comply with GDPR is to adopt a Privacy by Design approach to your systems.

Privacy by Design: Where Outcomes, Not Intentions, Matter

Privacy by Design is an approach to systems engineering that takes privacy into account throughout the whole engineering process.

It’s not about data protection per se.

Rather, the system is engineered in such a way that it doesn’t need protection.

The root principle is based on enabling service without having the client become identifiable or recognizable.

Three examples of Privacy by Design include:

  1. Dynamic Host Configuration Protocol (DHCP). With DHCP, a server maintains a pool of IP addresses, and randomly assigns an IP address to a device. Because the IP address is “leased” to a device, it doesn’t leak personal identifiers about the person using the device.
  2. Global Positioning System (GPS). A GPS device doesn’t require you to transmit data; rather, it relies on signals transmitted from GPS satellites whose positions are known. Without leaking your identity or location, it can provide you your geographic location.
  3. Radio-Frequency Identification (RFID). As it pertains to the Internet of Things (IoT), RFID can act as the bridge between the physical and digital world. The RFID tag is preregistered with the host system to establish identification. Then the tag communicates only by broadcasting its ID.

Zero-knowledge proof is one way you can implement Privacy by Design. It is a means of establishing proof by using something other than personal identifiers.

For example, a gambling website may use a Facebook sign-in, which can guarantee proof of age by asking Facebook.

In another example, a risqué game in the 1980s might ask questions about baseball players that only an older audience would know. Of course, the questions didn’t prevent a baseball-prodigy youngster from gaining access.

How Privacy by Design is achieved depends on the application, technologies and choice of approach. Daugherty can anonymize your data to protect you from penalties through GDPR. We can also analyze it to determine opportunities in the data where your organization can expand. Contact us today.