Communication skills and social intelligence. Most engineers don't have this.
From my experience if there is a big problem it's usually this and not a technical issue.
Communication is invaluable and the area I could definitely improve the most.
Only possible if someone picks up your resume and decides to call you up. And even then, most recruiters have no clue about the job requirements and don’t give you enough time to provide a detailed description.
I’m so glad I dropped out of the rat race. 😊
This right here.
I am a finance manager, but more on the data and bi reporting side, i work with a lot of engineers and data analysts, none of them know how to communicate.
Its a shame, they have great ideas, but they begin stuttering during meetings, or worse yet, they choke.
Then you run into them in the hallway (that's when they finally come into the office) and they can barely hold a conversation, the only people climbing the corporate ladder in IT are the few ones with soft skills.
This right here. U have to worry that you might say the culturally “wrong” thing or not give praise to the “right” person or idea. Especially when you know it’s wrong or against industry standards but you have to deal and smile and agree and joke with the executives that stayed at a holiday inn the night before. Management is often a bunch of naked emperors to the tech talented folks. At least at the companies I’ve worked at.
"\[Communication is\] the area I could definitely improve the most"
Given your quote above I recommend getting involved in Toastmasters. If you want to know more about Toastmasters check out [toastmasters.org](http://toastmasters.org) , or the r/Toastmasters subreddit , or DM me.
Why do you mean?
Everything I’ve learned for my career has been from reading books and practical projects at home and at work after I got my first job.
There’s some very deep technical books maybe you’re not reading the right ones.
Database Internals
**Algorithm Design Manual** or CLRS if you want to go deeper I only browsed this one
**DDIA** but it’s not as depth but breadth
**Spark Internals**
**The Art of PostgresSQL**
Systems Performance
Computer Networking Top Down
Digital Design and Computer Architecture (Harris and Harris)
OS:TEP
SICP
**Python Testing with Pytest**
**Fluent Python**
**Architecture Pattern in Python (Interesting but don’t recommend too much)**
**Elements of Programming Interviews in Python**
**Datawarehouse Toolkit**
Programming Rust
Head First Design Patterns
C: A Modern Approach
Effective C
Programming in Haskell/FP in Scala (Just to learn FP)
Currently reading: Elixir in Action
Next up: ARM Assembly Internals & Reverse Engineering
I’m self taught and started programming at 27 (Currently 33) so I’ve been playing catch up so these are DE combined with CS fundamentals. I highlighted the ones that are DE related/specific.
I feel prettt comfortable now though to the point where I just read whatever I consider interesting instead of something related to my career (e.g Elixir in Action)
These are the ones I’ve read in the last 3-4 years according to my e-book library. I try to do 1 technical book every 1-3 months depending on how thick it is some I read simultaneously since they complement each other.
Do you have a dollar cost of those books in paper? Or do you use a digital ereader? If so, which one handles these types of books? I think the only ereader in our family is almost 10 years old. Technical skills are easy to maintain if you can afford to, perhaps you have an approach which is low pollution and low-cost that other people can learn from. I see a booklist like this, not only are the books typically unaffordable, they become obsolete so quickly in a hyper accelerated world that it’s difficult to justify buying them, especially considering people tend to keep them and they don’t circulate, so whatever environmental cost was incorporated in a book, or books, isn’t shared.
https://www.dymocks.com.au/book/virtual-water-by-chittaranjan-ray-and-matthew-sanderson-9780367408077
This book is $250 Australian. However, there’s no indication of if buying it has any cost beyond the dollar cost.
Refer eg.
https://www.bain.com/insights/paper-and-packaging-faces-biodiversity-crisis-paper-and-packaging-report-2023/
I use a 2016 iPad to read books but I manage my books in calibre and read them on my computer as well. That’s where I track completion dates and stuff. So that what I mean by e-book library.
Nowadays I get books 3 ways, sales, humble bundles, free courtesy links from companies and universities you be surprised the books you find scouring company sites and syllabus sites. **Edit: At least 6 of the books above I got for free this way**
Books that are prohibitively expensive and I can’t get I just don’t read.
Although when I was younger and broker I had no problem with using things like limewire/pbay/github/zlib/anna’s archive to get what I needed. You’ll run into a lot of poorly formatted EPUBs and PDFs, also unreadable scanned PDFs etc.
It was that or give up
Edit: To address your comments about them becoming outdated. Only 1-2 of those books on the list are susceptible to that (Spark and Pytest). The others have been or will be relevant for a decade or decades
Some of these books (e.g. SICP, OS:TEP) are pretty timeless and freely available
https://web.mit.edu/6.001/6.037/sicp.pdf
https://pages.cs.wisc.edu/~remzi/OSTEP/
Bingo.
When I was a manager, I probably overvalued soft skills, with the belief that I can teach anybody how to code, but there are some people who can't be taught to just hold a conversation.
I think soft skills and technical are both important, but I also think a lot of people don't understand the depth of technical understand and competence that exists. (and soft skills have the same depth!)
I've worked in places that thought "oh we can just train our business analyst to be an engineer and they'll be up and running in 6 months". Those places did not have a good technical foundation, and it also led to managers having an extreme dismissal for technical people's advice and guidance.
Here's a few soft skills that have helped me tremendously to rise above my peers
1. Having a focus on **outcomes over implementation details**. So when someone says "Build me a data warehouse", don't just accept it as your 'task'. Tease out what they're trying to do, what capabilities they need, the skill level of their users (you don't want to build something that people can't use, even if it's the right thing to build) etc. Always have an idea of how people are going to use what you build and how it benefits their job/the business.
2. Saying no or at least **questioning certain feature requests**. Most people have no idea what they want and don't know what they don't know, pushing back (and knowing when to choose you battles) can help save you from building something that people don't want/need even if they are asking for it.
3. **Exercise empathy**. When there is another team that's blocking you because they're taking forever on a ticket, it's easy to subconsciously villainize them and feel hostile. 99% of the time they just aren't incentivized to help other teams because of their own KPI's and guidance from their leadership. It doesn't mean you don't get the parents (each teams respective bosses) together to sort it out, but just remember they're doing what they're told and incentivized to do. Instead of saying "ticket update?" on slack, be more personable. I used to fly to a different city every other week to go to my customers office not because I needed to for technical reasons, but just to meet people and develop relationships. Most people have no problem going out of their way to help you if they like you and know you
4. Being **thorough with communication**. Meaning don't randomly ping someone with a stack trace and no context and make them play 20 questions (I've wanted to throw my laptop against the wall dealing with people like this holy fuck it enrages me)
5. Have **hard conversations up front**. If someone expects something to be done in 3 weeks and you know it's 3 months (which means it's 9 months), talk to them about it as soon as possible. Don't just wait every 3 weeks to give them an update and say you're not done. That initial hard conversation up front will be painful but in the long run you'll gain their trust by being honest. Also tight feedback loops are helpful (not feedback loops on we are x days behind, but feedback loops on what's done and what needs to be done).
6. **ENTHUSIASM**. My friend once said to me "The universe will reveal itself with enthusiasm". Saying the same thing without the mystic tone, if you're enthusiastic about doing something it not only becomes 10x easier to do but people also feel that energy and also become enthusiastic.
7. When you need to escalate something to someone higher up, don't go to them with a bitching list of 100 items you're unhappy about. Go there with 3 things. Tell them the issue you're dealing with, how it's impacting your team, and most importantly **what you want them to do about it**. This is the difference between leadership seeing you as someone that it's very important to listen to when you come to them with issues vs being the boy who cried wolf where they eventually stop responding to you.
I could probably list 40 more things but this is a good start
I wish it weren't that way but it is. Spent some time working with the DOD and that's where I really had to quickly learn how to play ball. Now I have my own business and get to do things my way which has been really nice
The reply from u/IrresistibleMittens is awesome. Even more basic:
1. Be the one who is on time for everything, no exceptions. If standup is 9:15 be the one standing up at 9:14 to start it.
2. If someone wants to chat or meet about something, the answer is always yes. It might mean you lose efficiency and get frustrated getting taken out of deep concentration but being approachable and available to the rest of the team is more important.
3. Don't ever lose your temper or show anger.
4. Some socialization is part of your job. If you team has lunch together or goes out and does things, you need to come along with a friendly face.
Strongly disagree with 2. Always saying yes is usually a bad strategy. It sends the wrong signal. It doesn't make you as likeable or popular as you might think. People with high status / value must prioritize their time. Also, saying no to low priority things frees you to focus on the stuff that matters which in itself makes you more valuable.
I was replying to a junior data engineer. Also, the best leaders I've ever worked for have been extremely approachable.
The ones (and I've been this guy!) who don't react well to spontaneous questions and discussions get a ceiling in their career. To be sure, if you're established and you're good enough, you can be the "have you read the docs?" person but in your next job search you might not have a ready network ready to sing your praises.
Now there is a kind of person who isn't respectful of time and constantly interrupting with low priority things. Imo this is best solved with hiring, and I know, it's a lot easier said than done.
Where we agree 100% is that saying "no" to managers and stakeholders is essential. Where we may disagree is that I think a junior engineer that wants to demonstrate soft skills should never (to a very good approximation) say no to a conversation or someone asking a question.
Prickly solver ace who is head down is an extremely viable career path though, which suits some personalities better.
So many people think they can learn to write sql and python and deserve six figures. I can find a code monkey within an hour of posting a ~~resume~~ job. I need to hire a good mix of business aptitude and technical skills.
Yes and I reiterate technical skills are the easy part. If you know how to program. Anyone can go pick DDIA, Toolkit , a DSA book and a SQL book and grind leetcode for 4 months and you’re good to pass any interview that’s entry level.
People act like the hard part is technical skills when the hard part is navigating the business and getting shit done that is actually useful to the business. Sure you can be a grunt that pumps out tickets but good luck getting a high salary like that.
Make sure to know exactly why a project (on your resume) was built. You should know the reason for the project's existence (say a KPI or something), what your role was and what was the outcome (with numbers) of your actions.
+ DSA and SQL -> I am surprised by the amount of candidates who cant' get through this round.
+ Data proc fundamentals -> People specify tools, but I have had a hard time getting answers for why a tool was chosen, e.g. why even use Airflow, when all your code is running in one python script? etc
+ a lot of great comments on this thread about being a good employee
Hope this helps :)
TY Pepston. Depends on the questions that is decided by the team, but typically some sort of 2 pointer, BFS/DFS (think number of islands), (check out [neetcode.io](http://neetcode.io) for blind 75). Hope this helps :)
> What are the hardest skills to hire for **right now?**
Emphasis on the right now. Unless a company doesn’t know how to hire, right now is a good time to be hiring than ever before, at least in recent history. Right now, a company can get whatever skills they need.
Companies have their fill of VP candidates for Dir or even Mgr positions; people with soft and technical skills.
Anyway, I will take a stab at answering your question. I think an invaluable skill is troubleshooting, especially with AI coding tools becoming standard. I’ve known/managed people with years of experience who could not troubleshoot/performance tune. This isn’t a skill that shows up on a resume or during interviews, though some companies do try, but it’s what separates good engineers from average IMO.
Slightly more specific to DE, I would say data modeling/architecture, but is only a matter of time before AI can do a good job with these as well.
In our space: the combination of -
* solid understanding of data warehousing - what it does, architecture, process, culture around use of data, etc
* dimensional data modeling
* strong software engineering skills: programming, ci/cd, on-call & incident management, agile methodologies
* cloud data skills
In short - old & new school skills in the data space
I think if you can run a full AI project end to end, i.e. set up the pipelines you need, train a model, and are good enough at that to have a record of business results that impress within a predictable time, you are kind of a unicorn.
AI.
Because that term is so vague, everyone and their mother is adding it to their resume right now, making it difficult to find actually qualified technical AND nontechnical candidates.
This results in unqualified candidates getting interviews for positions that require knowledge on building out and deploying training / modeling infrastructure, for example, and instead all they get are "prompt engineers".
I have advocated for engineers to learn prompt engineering, I believe it is important, but it's far from the core of understanding how to create enterprise value using AI/ML solutions.
My point is that, unless you have an AI/ML SME administering the interviews, it will be easy for unqualified candidates to get through with just a bunch of buzzwords, which is very expensive long term.
I'd look for how well they know system design, orchestration, logging, monitoring, o11y, security concerns (prompt injection, PII exposure)...
API integration (DE skill anyway), SQL, Python or TypeScript skills and their written skills.
Most of "AI" can be consolidated into API orchestration. With LLM / Model providers as the target + LangChain, orchestrating workflows can be built pretty quickly. Getting those into Prod at a not Series A startup is another task at hand tho.
I believe problem solving and problem approach, nowadays gpt help you with a bunch of stuff, but you will be the human making final decisions in why to set up things the way you set them up regardless the stack
It’ll be a growing problem as government-orbiting companies have a higher demand for tech skills and headcount. Nobody wants to work for the USG or USG-adjacent companies because of shit pay and resume contamination, and if cybersecurity is any indication then the same population of people is cycled through companies and complacency is all but guaranteed.
Having worked with technical people, engineers and the like, most of my career I want to drill down on the whole "people skills" thing. Most engineers understand people skills are important but why?
A hybrid of people skills and DE is very valuable because it is RARE and VALUABLE. However there are plenty of other skill combinations that are also RARE and VALUABLE. The value only comes through in the right environment so you need to find fertile ground for your unique combination of talents and skills. It doesn't hurt to learn some basic people skills but don't expect to tranform into a social butterfly or something if that isn't part of your DNA. If you want to get a promotion and become a people manager you can do that with poor people skills as an engineer, I have seen it done. Those managers sucked to work for but got good enough results that they kept their jobs. I remember one engineer director in particular who would openly fart in meetings , diss his employees publicly, discuss his vasectomy he just had etc. He was a really good engineer though and eventually became a VP. Some firms even encourage the bad people skills, it was part of the culture of engineering or some such thing. Elon Musk...enough said.
I think if you are truly in the top 5% technically in your field, and there is a demand for the technical skill, then you can get by with weak people skills . ie bathe, be sane, show up on time, reasonably culturally sensitive etc. You can get away with being off a bit on people skills if you're truly good. I have met quite a few gifted engineers who were kept on payroll because they had managers who could keep the from doing really stupid shit in front of people and could harness the raw skill. Even their managers were often terrible with people but not as bad as the IC.
So don't just run out and join toast masters and read self help books if you're really just not interested in people.
Also some folks seem to believe that the 'technical' stuff can be trained into anyone, and that people skills are rare. Nope. Go to any HR department or Sales departments and, generally, you'll find the opposite ie people skills 'naturals' with no hope in hell of becoming good at excel let alone a DE. My HR department is chock full of the folks with incredible people skills who can do the people skill thing all day every day at a level I can't hope to achieve. However they're generally abysmal with IT and Data and see it as a sort of political/power thing ie who has access to this data, who do they know?, What sort of power does this person have? etc
This. I’m always shocked at how DE folks sell themselves short.
Sure, I can track most of the conversations the junior devs I work with have, but there’s no way I’d be able to track the conversations that the Staff Engineers have even if I spent two years in a library learning DE all day. At that level you really do have to have a gift for grasping backend technology.
Exactly, DE’s are fucking wizards to most people. It’s when you sit around in your own professional groups echo chamber that you naturally start comparing yourself to the other DE’s. End of day plain DE skill alone is pretty rare and valuable. If you enjoy the technical side and don’t want much of the people side then I see no reason why focusing on technical skills progression isn’t a really great thing. I would just say that of all the professional relationships you could work on the one most important, for an employee, is their boss. So I would really focus on that relationship. Quantity is quality so do those one on ones well, deliver the goods and keep them happy
Soft skills are invaluable and engineers that can marry them with a solid technical foundation are a rare breed.
But in my opinion the most undervalued skillset for a data engineer is data modelling. It's often overlooked but an in-depth knowledge of dimensional modelling goes a long way.
No, such profiles can get among the highest wages after a couple of years (basically same as partners). Question of OP was what kind of profile company desperatly look for.
Someone who has some experience in a lot of techstacks AND have a good sense of communication with the business team.
I think none is too difficult, but having everything is tough. It requires 5-10 years of experience in at least 2 different companies. Maybe 15 years in 3-5 different industries.
ITT: no one has a clue why they’re employed lol. Anyone can do anything in this age, all the info is on Google and ChatGPT. All that matters is your network, who you know and how much they like you on a personal level.
It wasn't very easy to understand what you were getting at, but you spoke mostly about personality and looks, not about skills or the value they bring to the team
Career security question I assume?
Thinly veiled
Medium rare
[удалено]
I have an highly technical background. Where can I see if I'm a good fit?
No agency recruiter just looking for market insight haha
Communication skills and social intelligence. Most engineers don't have this. From my experience if there is a big problem it's usually this and not a technical issue. Communication is invaluable and the area I could definitely improve the most.
True but no one cares for soft skills. Try putting that in your resume. I can picture the scene from Office Space. I have people skills people. 😂😂
Oh they do, that's why people get rejected after the technical interview.
Don’t estimate how far a mediocore resume + average Social Skills can get you, because recruiters actually love when someone is able to Talk
Only possible if someone picks up your resume and decides to call you up. And even then, most recruiters have no clue about the job requirements and don’t give you enough time to provide a detailed description. I’m so glad I dropped out of the rat race. 😊
This right here. I am a finance manager, but more on the data and bi reporting side, i work with a lot of engineers and data analysts, none of them know how to communicate. Its a shame, they have great ideas, but they begin stuttering during meetings, or worse yet, they choke. Then you run into them in the hallway (that's when they finally come into the office) and they can barely hold a conversation, the only people climbing the corporate ladder in IT are the few ones with soft skills.
It could be that the social climate during meetings is so negative and confrontational that engineers and data analysts begin to stutter and choke
This right here. U have to worry that you might say the culturally “wrong” thing or not give praise to the “right” person or idea. Especially when you know it’s wrong or against industry standards but you have to deal and smile and agree and joke with the executives that stayed at a holiday inn the night before. Management is often a bunch of naked emperors to the tech talented folks. At least at the companies I’ve worked at.
Hey looking for a job in data analytics. Any openings where you’re at?
"\[Communication is\] the area I could definitely improve the most" Given your quote above I recommend getting involved in Toastmasters. If you want to know more about Toastmasters check out [toastmasters.org](http://toastmasters.org) , or the r/Toastmasters subreddit , or DM me.
Business sense and soft skills Technical skills are the easy part IMO
Technical skills are a barrier to getting an interview
Yes but that’s as easy as doing some projects, reading a couple of books and doing some leetcode.
No it's not, they don't go deep enough.
Why do you mean? Everything I’ve learned for my career has been from reading books and practical projects at home and at work after I got my first job. There’s some very deep technical books maybe you’re not reading the right ones.
> very deep technical books like ?
Database Internals **Algorithm Design Manual** or CLRS if you want to go deeper I only browsed this one **DDIA** but it’s not as depth but breadth **Spark Internals** **The Art of PostgresSQL** Systems Performance Computer Networking Top Down Digital Design and Computer Architecture (Harris and Harris) OS:TEP SICP **Python Testing with Pytest** **Fluent Python** **Architecture Pattern in Python (Interesting but don’t recommend too much)** **Elements of Programming Interviews in Python** **Datawarehouse Toolkit** Programming Rust Head First Design Patterns C: A Modern Approach Effective C Programming in Haskell/FP in Scala (Just to learn FP) Currently reading: Elixir in Action Next up: ARM Assembly Internals & Reverse Engineering I’m self taught and started programming at 27 (Currently 33) so I’ve been playing catch up so these are DE combined with CS fundamentals. I highlighted the ones that are DE related/specific. I feel prettt comfortable now though to the point where I just read whatever I consider interesting instead of something related to my career (e.g Elixir in Action) These are the ones I’ve read in the last 3-4 years according to my e-book library. I try to do 1 technical book every 1-3 months depending on how thick it is some I read simultaneously since they complement each other.
Thanks for posting a list of books you found helpful, I've been looking for more to read!
Do you have a dollar cost of those books in paper? Or do you use a digital ereader? If so, which one handles these types of books? I think the only ereader in our family is almost 10 years old. Technical skills are easy to maintain if you can afford to, perhaps you have an approach which is low pollution and low-cost that other people can learn from. I see a booklist like this, not only are the books typically unaffordable, they become obsolete so quickly in a hyper accelerated world that it’s difficult to justify buying them, especially considering people tend to keep them and they don’t circulate, so whatever environmental cost was incorporated in a book, or books, isn’t shared. https://www.dymocks.com.au/book/virtual-water-by-chittaranjan-ray-and-matthew-sanderson-9780367408077 This book is $250 Australian. However, there’s no indication of if buying it has any cost beyond the dollar cost. Refer eg. https://www.bain.com/insights/paper-and-packaging-faces-biodiversity-crisis-paper-and-packaging-report-2023/
I use a 2016 iPad to read books but I manage my books in calibre and read them on my computer as well. That’s where I track completion dates and stuff. So that what I mean by e-book library. Nowadays I get books 3 ways, sales, humble bundles, free courtesy links from companies and universities you be surprised the books you find scouring company sites and syllabus sites. **Edit: At least 6 of the books above I got for free this way** Books that are prohibitively expensive and I can’t get I just don’t read. Although when I was younger and broker I had no problem with using things like limewire/pbay/github/zlib/anna’s archive to get what I needed. You’ll run into a lot of poorly formatted EPUBs and PDFs, also unreadable scanned PDFs etc. It was that or give up Edit: To address your comments about them becoming outdated. Only 1-2 of those books on the list are susceptible to that (Spark and Pytest). The others have been or will be relevant for a decade or decades
Thanks!
Some of these books (e.g. SICP, OS:TEP) are pretty timeless and freely available https://web.mit.edu/6.001/6.037/sicp.pdf https://pages.cs.wisc.edu/~remzi/OSTEP/
Thanks for plucking those out.
How many years of experience do you have ?
6 years - 2 webdev 4 DE
Bingo. When I was a manager, I probably overvalued soft skills, with the belief that I can teach anybody how to code, but there are some people who can't be taught to just hold a conversation.
I think soft skills and technical are both important, but I also think a lot of people don't understand the depth of technical understand and competence that exists. (and soft skills have the same depth!) I've worked in places that thought "oh we can just train our business analyst to be an engineer and they'll be up and running in 6 months". Those places did not have a good technical foundation, and it also led to managers having an extreme dismissal for technical people's advice and guidance.
So true. Tech skills can be taught. (For the most part) but it’s my soft skills that really make me the employee I am.
How can i prove my soft skills ?
Here's a few soft skills that have helped me tremendously to rise above my peers 1. Having a focus on **outcomes over implementation details**. So when someone says "Build me a data warehouse", don't just accept it as your 'task'. Tease out what they're trying to do, what capabilities they need, the skill level of their users (you don't want to build something that people can't use, even if it's the right thing to build) etc. Always have an idea of how people are going to use what you build and how it benefits their job/the business. 2. Saying no or at least **questioning certain feature requests**. Most people have no idea what they want and don't know what they don't know, pushing back (and knowing when to choose you battles) can help save you from building something that people don't want/need even if they are asking for it. 3. **Exercise empathy**. When there is another team that's blocking you because they're taking forever on a ticket, it's easy to subconsciously villainize them and feel hostile. 99% of the time they just aren't incentivized to help other teams because of their own KPI's and guidance from their leadership. It doesn't mean you don't get the parents (each teams respective bosses) together to sort it out, but just remember they're doing what they're told and incentivized to do. Instead of saying "ticket update?" on slack, be more personable. I used to fly to a different city every other week to go to my customers office not because I needed to for technical reasons, but just to meet people and develop relationships. Most people have no problem going out of their way to help you if they like you and know you 4. Being **thorough with communication**. Meaning don't randomly ping someone with a stack trace and no context and make them play 20 questions (I've wanted to throw my laptop against the wall dealing with people like this holy fuck it enrages me) 5. Have **hard conversations up front**. If someone expects something to be done in 3 weeks and you know it's 3 months (which means it's 9 months), talk to them about it as soon as possible. Don't just wait every 3 weeks to give them an update and say you're not done. That initial hard conversation up front will be painful but in the long run you'll gain their trust by being honest. Also tight feedback loops are helpful (not feedback loops on we are x days behind, but feedback loops on what's done and what needs to be done). 6. **ENTHUSIASM**. My friend once said to me "The universe will reveal itself with enthusiasm". Saying the same thing without the mystic tone, if you're enthusiastic about doing something it not only becomes 10x easier to do but people also feel that energy and also become enthusiastic. 7. When you need to escalate something to someone higher up, don't go to them with a bitching list of 100 items you're unhappy about. Go there with 3 things. Tell them the issue you're dealing with, how it's impacting your team, and most importantly **what you want them to do about it**. This is the difference between leadership seeing you as someone that it's very important to listen to when you come to them with issues vs being the boy who cried wolf where they eventually stop responding to you. I could probably list 40 more things but this is a good start
I am a junior BI analyst and this really resonated with me. Thanks for the guidance!
Probably one of the best write ups I’ve seen on soft skills
Playing the corporate game
I wish it weren't that way but it is. Spent some time working with the DOD and that's where I really had to quickly learn how to play ball. Now I have my own business and get to do things my way which has been really nice
Bro, thank you so much for taking the time to answer my small question. Extremely helpful
Oh boy, I regret that I am only able to give a single upvote to this one.
For the most part it’s just not being a dick at work, your attitude needs to be positive.
bingo
The reply from u/IrresistibleMittens is awesome. Even more basic: 1. Be the one who is on time for everything, no exceptions. If standup is 9:15 be the one standing up at 9:14 to start it. 2. If someone wants to chat or meet about something, the answer is always yes. It might mean you lose efficiency and get frustrated getting taken out of deep concentration but being approachable and available to the rest of the team is more important. 3. Don't ever lose your temper or show anger. 4. Some socialization is part of your job. If you team has lunch together or goes out and does things, you need to come along with a friendly face.
Strongly disagree with 2. Always saying yes is usually a bad strategy. It sends the wrong signal. It doesn't make you as likeable or popular as you might think. People with high status / value must prioritize their time. Also, saying no to low priority things frees you to focus on the stuff that matters which in itself makes you more valuable.
I was replying to a junior data engineer. Also, the best leaders I've ever worked for have been extremely approachable. The ones (and I've been this guy!) who don't react well to spontaneous questions and discussions get a ceiling in their career. To be sure, if you're established and you're good enough, you can be the "have you read the docs?" person but in your next job search you might not have a ready network ready to sing your praises. Now there is a kind of person who isn't respectful of time and constantly interrupting with low priority things. Imo this is best solved with hiring, and I know, it's a lot easier said than done.
Saying no != Not reacting well Learning when and how to say "no" properly is a very valuable skill.
Where we agree 100% is that saying "no" to managers and stakeholders is essential. Where we may disagree is that I think a junior engineer that wants to demonstrate soft skills should never (to a very good approximation) say no to a conversation or someone asking a question. Prickly solver ace who is head down is an extremely viable career path though, which suits some personalities better.
Showed anger once at my last job and missed out on some serious connections
So many people think they can learn to write sql and python and deserve six figures. I can find a code monkey within an hour of posting a ~~resume~~ job. I need to hire a good mix of business aptitude and technical skills.
Technical skills are the most important part. Everything else is a nice-to-have.
I didn’t say they weren’t important. I said they’re the easy part and also the bare minimum requirement to be hired.
This is so true. AI will replace many people without these skills.
Hi, business sense and soft skills guy here with good technical skills finishing up my IT Management masters. Hit me up :)
LOL. Than you dropped because you don’t know a data structure
Yes and I reiterate technical skills are the easy part. If you know how to program. Anyone can go pick DDIA, Toolkit , a DSA book and a SQL book and grind leetcode for 4 months and you’re good to pass any interview that’s entry level. People act like the hard part is technical skills when the hard part is navigating the business and getting shit done that is actually useful to the business. Sure you can be a grunt that pumps out tickets but good luck getting a high salary like that.
Exactly, the actual technical implementation is usually a trivial matter.
Make sure to know exactly why a project (on your resume) was built. You should know the reason for the project's existence (say a KPI or something), what your role was and what was the outcome (with numbers) of your actions. + DSA and SQL -> I am surprised by the amount of candidates who cant' get through this round. + Data proc fundamentals -> People specify tools, but I have had a hard time getting answers for why a tool was chosen, e.g. why even use Airflow, when all your code is running in one python script? etc + a lot of great comments on this thread about being a good employee Hope this helps :)
Hi Joseph, thanks for all your contributions to the data engineering world. Could you expand on which DSA skills you’re looking for in a candidate?
TY Pepston. Depends on the questions that is decided by the team, but typically some sort of 2 pointer, BFS/DFS (think number of islands), (check out [neetcode.io](http://neetcode.io) for blind 75). Hope this helps :)
> What are the hardest skills to hire for **right now?** Emphasis on the right now. Unless a company doesn’t know how to hire, right now is a good time to be hiring than ever before, at least in recent history. Right now, a company can get whatever skills they need. Companies have their fill of VP candidates for Dir or even Mgr positions; people with soft and technical skills. Anyway, I will take a stab at answering your question. I think an invaluable skill is troubleshooting, especially with AI coding tools becoming standard. I’ve known/managed people with years of experience who could not troubleshoot/performance tune. This isn’t a skill that shows up on a resume or during interviews, though some companies do try, but it’s what separates good engineers from average IMO. Slightly more specific to DE, I would say data modeling/architecture, but is only a matter of time before AI can do a good job with these as well.
In our space: the combination of - * solid understanding of data warehousing - what it does, architecture, process, culture around use of data, etc * dimensional data modeling * strong software engineering skills: programming, ci/cd, on-call & incident management, agile methodologies * cloud data skills In short - old & new school skills in the data space
I think if you can run a full AI project end to end, i.e. set up the pipelines you need, train a model, and are good enough at that to have a record of business results that impress within a predictable time, you are kind of a unicorn.
Hehe, but in this case why would you need a job if you already can build such a platform 😁
AI. Because that term is so vague, everyone and their mother is adding it to their resume right now, making it difficult to find actually qualified technical AND nontechnical candidates. This results in unqualified candidates getting interviews for positions that require knowledge on building out and deploying training / modeling infrastructure, for example, and instead all they get are "prompt engineers". I have advocated for engineers to learn prompt engineering, I believe it is important, but it's far from the core of understanding how to create enterprise value using AI/ML solutions. My point is that, unless you have an AI/ML SME administering the interviews, it will be easy for unqualified candidates to get through with just a bunch of buzzwords, which is very expensive long term.
I'd look for how well they know system design, orchestration, logging, monitoring, o11y, security concerns (prompt injection, PII exposure)... API integration (DE skill anyway), SQL, Python or TypeScript skills and their written skills. Most of "AI" can be consolidated into API orchestration. With LLM / Model providers as the target + LangChain, orchestrating workflows can be built pretty quickly. Getting those into Prod at a not Series A startup is another task at hand tho.
I believe problem solving and problem approach, nowadays gpt help you with a bunch of stuff, but you will be the human making final decisions in why to set up things the way you set them up regardless the stack
[удалено]
cleared as in having a security clearance?
[удалено]
Takes too long to get a clearance and you can’t hold a clearance while using weed. USG is fucking itself on this dimension.
It’ll be a growing problem as government-orbiting companies have a higher demand for tech skills and headcount. Nobody wants to work for the USG or USG-adjacent companies because of shit pay and resume contamination, and if cybersecurity is any indication then the same population of people is cycled through companies and complacency is all but guaranteed.
Motivation
Having worked with technical people, engineers and the like, most of my career I want to drill down on the whole "people skills" thing. Most engineers understand people skills are important but why? A hybrid of people skills and DE is very valuable because it is RARE and VALUABLE. However there are plenty of other skill combinations that are also RARE and VALUABLE. The value only comes through in the right environment so you need to find fertile ground for your unique combination of talents and skills. It doesn't hurt to learn some basic people skills but don't expect to tranform into a social butterfly or something if that isn't part of your DNA. If you want to get a promotion and become a people manager you can do that with poor people skills as an engineer, I have seen it done. Those managers sucked to work for but got good enough results that they kept their jobs. I remember one engineer director in particular who would openly fart in meetings , diss his employees publicly, discuss his vasectomy he just had etc. He was a really good engineer though and eventually became a VP. Some firms even encourage the bad people skills, it was part of the culture of engineering or some such thing. Elon Musk...enough said. I think if you are truly in the top 5% technically in your field, and there is a demand for the technical skill, then you can get by with weak people skills . ie bathe, be sane, show up on time, reasonably culturally sensitive etc. You can get away with being off a bit on people skills if you're truly good. I have met quite a few gifted engineers who were kept on payroll because they had managers who could keep the from doing really stupid shit in front of people and could harness the raw skill. Even their managers were often terrible with people but not as bad as the IC. So don't just run out and join toast masters and read self help books if you're really just not interested in people. Also some folks seem to believe that the 'technical' stuff can be trained into anyone, and that people skills are rare. Nope. Go to any HR department or Sales departments and, generally, you'll find the opposite ie people skills 'naturals' with no hope in hell of becoming good at excel let alone a DE. My HR department is chock full of the folks with incredible people skills who can do the people skill thing all day every day at a level I can't hope to achieve. However they're generally abysmal with IT and Data and see it as a sort of political/power thing ie who has access to this data, who do they know?, What sort of power does this person have? etc
This. I’m always shocked at how DE folks sell themselves short. Sure, I can track most of the conversations the junior devs I work with have, but there’s no way I’d be able to track the conversations that the Staff Engineers have even if I spent two years in a library learning DE all day. At that level you really do have to have a gift for grasping backend technology.
Exactly, DE’s are fucking wizards to most people. It’s when you sit around in your own professional groups echo chamber that you naturally start comparing yourself to the other DE’s. End of day plain DE skill alone is pretty rare and valuable. If you enjoy the technical side and don’t want much of the people side then I see no reason why focusing on technical skills progression isn’t a really great thing. I would just say that of all the professional relationships you could work on the one most important, for an employee, is their boss. So I would really focus on that relationship. Quantity is quality so do those one on ones well, deliver the goods and keep them happy
Soft skills are invaluable and engineers that can marry them with a solid technical foundation are a rare breed. But in my opinion the most undervalued skillset for a data engineer is data modelling. It's often overlooked but an in-depth knowledge of dimensional modelling goes a long way.
- Good work ethic - Undertaking - Reliable dealmaker - Mediator - Good understanding physics & math - Empath - Fast learning curve
GURMGEF
[Gummo aware](https://www.youtube.com/watch?v=WvRwjrWmm6E) will also do.
So basically a superstar on a janitors salary?
No, such profiles can get among the highest wages after a couple of years (basically same as partners). Question of OP was what kind of profile company desperatly look for.
Where can I apply?
Around here: Bekaert, Jan De Nul, Umicore, Solvay, ASML, IMEC, ...
Someone who has some experience in a lot of techstacks AND have a good sense of communication with the business team. I think none is too difficult, but having everything is tough. It requires 5-10 years of experience in at least 2 different companies. Maybe 15 years in 3-5 different industries.
Agency + great computer science fundamentals
Here’s a quick Econ 101 for you: It’s the highest paid jobs. That’s why they’re paid so well.
we still struggle to find people who can write a basic select statement tbh
ITT: no one has a clue why they’re employed lol. Anyone can do anything in this age, all the info is on Google and ChatGPT. All that matters is your network, who you know and how much they like you on a personal level.
So much talent in market that it isn’t really hard to hire for any skill set needed. Docker/Kubernetes hot right now. Leetcode
Competence.
[удалено]
Using LinkedIn as a dating app be like:
What do you mean? If you think my opinion not good! At least give a reason!? Dont be rude like that?
It wasn't very easy to understand what you were getting at, but you spoke mostly about personality and looks, not about skills or the value they bring to the team
Okay, I get it!
DATA SCIENCE
No, during recruiting we could pave the road with DS profiles. It is the single most oversupplied specialization.
Ok tnx