How to stand out as a data engineer
Standing out as a data engineer
- Introduction
- The basic role of a data engineer
- The anti-pattern
- What this leads to
- A shift to focusing on business outcomes
- Requiring a consulting mindset
- Conclusion
Introduction
Whether it’s a desire to stand out in a competitive job market, receive a promotion, or just personal sense of achievement, we all want to stand out as a data engineer that has that special something that makes them more effective than others.
Oftentimes in the pursuit of this goal, aspiring or junior engineers take a bunch of courses to become proficient in the hot new tools. They may read books on fundamentals of data engineering to better understand data modeling, ETL, or distributed processing.
While these can all be helpful tools, I have found that the best way to stand out as a data engineer is a singular focus on the business
The basic role of a data engineer
A data engineer’s job is to create and maintain data systems that collect, store, and serve data.
The anti-pattern
What data engineers should avoid at all cost is being seen as a tool to build these data systems. This is a data engineer who is focused on tooling and how to tweak their systems to incorporate the newest technology. This engineer is given requirements by business stakeholder and executes them faithfully. Once they get their requirements they disappear for a few weeks before returning triumphantly with a newly constructed data pipeline. This engineer hands off their new pipeline and moves on to the next project.
This may sound like a great engineer. They are staying on top of their field, efficiently tackling tickets, and trying to make improvements to their pipeline.
What this leads to
The problem with the above approach is that it ignores the true purpose of the data engineer – to solve business problems. Business stakeholders are generally not experts in data. The requirements they give to a data engineer may not accurately reflect what is most needed by the business. Without
A shift to focusing on business outcomes
At the end of the day, the only purpose of the data engineer is to increase the company’s revenue or decrease their costs. Everything else is a distraction to these main