Why should you focus on augmented data management
As time goes on and the volume of data rises, traditional data management processes aren’t enough. It gets more complicated and time-consuming to draw out valuable data, process it, and prepare it for other data-related tasks. But, with the development of AI, we see the possibilities of augmented data management. Traditional approaches are falling somewhat […]
How to manage the pull of data gravity
If you are an enterprise company or an entity that either generates or collects enormous amounts of data, then you must be familiar with the fact that the more data you have, the more you’ll add on. As soon as you recognize the value of data in your everyday operations, you will see that data […]
Data fabric and its tech details
Welcome to the final post in the data fabric series. So far we introduced data fabric and talked about the business benefits it brings. Now it is time to look at technical questions that may come to mind as you explore the concept of the data fabric. Integration With Existing Data Systems Unless you are […]
Promoting data democratization and governance with data fabric
Lately, our colleague Valentina Dugan created a great post explaining How to tackle data security and concerns. Some key terms she covers there are unauthorized access, compliance, data masking, and data democratization. I couldn’t have imagined a more perfect backdrop for data fabric, the topic I’ve been wanting to write about for the past few […]
How to tackle data security and concerns
In these days of massive data amounts, AI, the development of online media, and such, data privacy and personal data have become an all-time concern. Alongside people that try to test the boundaries of data security and put such information at harm, even data systems and infrastructure can show weaknesses in their defenses. But what […]
Get to know our data detectives – Wilim
Come and meet Wilim, our data detective and data engineer. With a name so unique, you can bet his data skills are also unique. Dive deep into what he loves about his works, what motivates him, and why he believes data engineering brings so much fun and creativity. Tell us who you are and what […]
Data ethics and why it’s important for brand trustworthiness
There isn’t a blog post nowadays without someone saying this: “All companies are data companies.” And it’s absolutely true. Data has become the pillar of business society and something people trade with. Information in the digital age has become somewhat of a gold mine. But, with each piece of information comes the hot topic of […]
Data observability in a nutshell
There is a lot of talk about broad aspects of data science and data engineering, but few mention the importance of quality data and the processes behind it. Data observability is a term that handles data behind the scenes. From infrastructure to data movement, it’s integral to provide an unobstructed flow of information. Data observability […]
Apache Spark – How to create a powerful streaming application
After setting up Apache Kafka, the next step in our Retail Business Intelligence Platform project was to set up Spark, another widely used software solution from the Apache workshop. Apache Spark, the real spark necessary to ignite our project and turn it into a true stream processing arrangement, is probably the most famous streaming engine […]
How to setup a Kafka Streaming project
If you’ve read the previous blog post about this project, you should already know what was the main idea of the project. We’re trying to estimate the unit sales of more than 3000 distinct items in 10 different Walmart stores. The dataset provides us with more than 5 years of data about the dates on […]