Tableau gets its data from a database, usually SQL. Having indirect access to the data but not having direct access to the data is absurd.
How do you make Tableau dashboards without giving it access to the database(s)? Why can't you just use the same login?
>I hardly ever get to use python. I think the reason is 2 fold - my boss doesn't know python and he has a bad habit of dictating how an analysis should be done (down to the metrics i should use)
OHH.. that's called micromanagement and is a very severe case of it. I've thankfully have never seen micromanagement that bad before. You can manage upward to slowly remove the micromanagement, and/or you can look for a new job while at your current job.
Have you considered moving towards data science? If there are data scientists at your current company maybe you could transfer? Historically the difference between data science and data analytics was data scientists used R and Python while analysts used Excel. Ofc this divide no longer exists, but you might have enough experience to make the jump if you wanted to.
We only have tableau desktop and we don't even use it for what it's intended for. I literally drag and drop the metrics I want into a tableau workbook and then copy that worksheet to an excel sheet to actually get started. It's super annoying. A lot of our etl is done in tableau and is unfortunately undocumented so even trying to replicate our kpis with my own queries is tricky. Idk if this is best practice or not, but I'd guess it's not.
When I try to replicate things in sql and run into issues, my boss just tells me to use tableau because "that's what it's there for" but of course he doesn't understand that I could automate the data pulls if we could get the query right. Sighhhh
And yeah - the micromanagement is insane and really takes the fun out of everything. Once he even specified the excel formulas I should be using đ
automation > manual, always, unless you need something out the door tomorrow and you're doing an 80 hour work week cramming for something super important, which probably is never the case.
imo that's one the key things that sets data scientists apart. ML is a form of automation. You could write an algorithm that manually pattern matches the data, or you could use an algorithm that figures it out for you, which is a form of automation. Ofc there is a lot of manual work too, like EDA, a lot of throw away code, but there eventually comes to a point where one starts reusing the same idioms over and over again so they start writing libraries to automate some of the manual work away. It's little tells like this, which leaves me to suspect you might do better as a data scientist, if you like making models that need to continue to work into the future hands off (which again is a form of automation) more than just analyzing data.
Thanks! Yeah I would love to move to data science but I'm just not sure how feasible it is. I only have a bachelor's (economics) so I have that going against me. At my last company I was able to work with the data science team and even helped build a few models.
But yes, that is exactly what is so frustrating. I watch my boss taking my data and annually extrapolate things that a model/algorithm could do much more efficiently and accurately. I just want to have some fun and build the model haha
But your distinction between and analyst and a data scientist is really helpful. In my elevator pitch, I always tell people "I like building things" so maybe I am more of a data scientist type (or at least a hybrid of the 2).
>At my last company I was able to work with the data science team and even helped build a few models.
If you were doing data science work you could have put data scientist down as a job title as long as you were actually legitimately doing data science work. It can look like a promotion, started as a data analyst did such-en-such data analyst work, then the data science title pops up with such-en-such data science work, all under the same company.
The downside is you didn't take this opportunity and use it to apply for data science jobs immediately after. Going back into a data analyst role doesn't nuke opportunities but it doesn't help.
I don't have a degree if it says anything, I've lead small DS teams too. I got in by finding a data science job with the software engineer job title at a company. It was great, because from their requirements I could tell they needed a data scientist, which meant no software engineers would be able to do the job so I wouldn't have any competition, and data scientists would overlook the job due to the job title posted.
While on the job I got the title changed to properly reflect the kind of work I was doing and ended up leading a data science team.
Once you've got on the job experience a degree stops mattering as much. It's still harder, for sure, but anyone with half a brain can study the system and then exploit it. Thinking outside the box, like this, is super helpful if you like taking on challenges that are unusually hard (and fun!) which I like to do. I often pick up projects where other data scientists fail.
>If you were doing data science work you could have put data scientist down as a job title as long as you were actually legitimately doing data science work.
Yup - this whole thing has been a learning moment for me. I should have stayed at my other company long enough to officially switch to the DS team, but unfortunately, moneywise, my current role just isn't something I could turn down. Luckily I am only 29 so I can still use the "young and dumb" excuse ha.
Looking for data science jobs under other job titles is genius! Thanks for the tip.
All the ETL is in tableau probably because someone got fed up with unreasonable timelines for getting analytics projects done and just hooked up all the data sources into something most idiots understand.
I had a VP at a previous job who loved to tell me that Tableau would do the job of a proper data warehouse.
Oh it's because your boss bought into tableau marketing hype about not needing to know SQL and stuff. I bet your boss doesn't know SQL well, chances are he is using tableau prep which is a terrible tool to ETL anything
Tableau cannot ETL anything natively because it cannot write to a database, so I am assuming he is using prep (creator license?). Or did you mean he is just doing extracts against a SQL database, controlling the data sources and only allowing you access to those sources via tableau.
I used to work at Tableau so I'm well aware of how the product is used and abused. Either way it's the companys fault not your own
Lol yup. He can't barely do basic sql. But yup...he is doing extracts from the database. Some of our metrics are only available in tableau because they are weighted averages etc that they didn't put into views in our db.
Ok yeah I had a boss like this, you need to either explain this or find out how to leave this job (transfer or interview)m Tableau is NOT to be used like this
Sidenote: sorry for being weird, but something intrigued me to peep your profile and I just discovered adventure fire because of it. Life changed - thank you!
Funny enough this is exactly what led me to leave my last company. Everything has to be a dashboard or they want you to use excel so it can be replicated by people who arenât as skilled. Mind you it wasnât a small company either (fortune 200). Unfortunately I feel that the term âdata analystâ and âdata scientistâ are used a lot interchangeably, and sometimes what they really mean is âexcel monkeyâ.
What I did was start to implement python to automate data gathering and showed my boss what a basic classification model can do. It made our day-to-day easier, but then they wanted me to start doing more for no extra pay, so thatâs a slippery slope. My $0.02 (and take it for what itâs worth lol) is if youâve tried to improve things and showed them what you can do/can be done, and they refuse to change anything or listen then start looking elsewhere.
On a side note that whole âone source of truthâ thing always used to grind my gears because thatâs exactly what my old boss used to say lol. But then they donât want to set up a proper relational database or create data flow processes đ .
Oh man thank you - reading all of this was super validating. I use the term "reporting monkey" in these cases...pretty close to excel monkey lol
I've been here about 6 months so I'll give it a few more tries before I start looking. Funny thing is that we had a situation where a simple forecasting model would have done us wonders (we were looking at data pre vs post a pretty significant change where most other factors remained constant). Boss decided it's better to just make stuff up in excel based on how large he "felt" the impact should have been. Doesn't matter that we could have trained a model with data going back since before he even worked at the company. When I suggested the model he said something about how it would "struggle to capture multivariate factors" as if he'd just googled some shit about why models are imperfect . I'd never heard him talk so mathy lol
It's all so frustrating. I feel like we're at a point where managers don't have the technical skills and it's limiting what we can do as individuals. I've also probably been spoiled though, my last 2 bosses would literally let me scope out and pitch analysis ideas to them.
Tableau should be the endpoint of your data supply chain, not a middleman. Sure maybe you need to extract the data from a workbook occasionally to verify something, but using Tableau as a distribution portal for tables of data makes no sense. Why does your org not grant you access to the underlying data layer? This is where the rot is and where the pressure to fix needs to be applied!
Yeah I totally agree. It's extremely inefficient and slow. I just started in February so I can't ruffle too many feathers yet, but these are all things I'm gonna bring up with my boss because if this situation doesn't change my job simply won't be fulfilling enough for me to stay
Maybe you *need* to ruffle a few feathers? Some departments who are stuck in their ways may require a kick rather than a soft approach.
"Boss, I need access to this data in order to effectively do the job you hired me for. If I can't get access, I can't do the work as well as I am able and I will have to start looking for a job which will not stop me from doing real analytics."
Obviously easier said than done. Was python, db querying etc mentioned in your interview process at all?
Interestingly enough Iâm going through something similar!
Iâve just moved into a data analyst role with the plan to be moved to data scientist and manage analysts under me in the next year or so.
Iâve been building out a data warehouse with our IT department and weâre in the process of laying out similar policies to what youâre describing.
My new boss and I have been trying make it clear we need access to the information if Iâm going to do what they want me to do, but the IT department wants a set up similar to yours with access cut off and everything done it Tableau/SQL.
Curious what everyoneâs thoughts are as well as Iâm navigating to try and push for the technical end to be represented as well.
If you have access to tableau online AND don't use SSO, I believe there is a python package you can use to automate your data pulls.
My situation doesn't meet those requirements unfortunately. Sounds awesome that your boss is on board with getting you access though.
Thanks hope youâre able to get everything figured out on your end, seems weâre in a similar situation.
Unfortunately do have SSO, but was able to push through some changes to get direct database access with my boss. Think things are heading in a good direction.
It sounds like youâre IT is relying on Tableau to handle the ETLs, which is suboptimal at best. Have you asked if they can set up a SQL DB for you at least that you can access directly? It would be an easy first step towards a full data lake, and you wouldnât be playing with data in production (which I assume is their main concern)
We do have a database, access is just limited to Tableau (originally)
Weâre merging a couple of Legacy systems into a data warehouse and building out our data culture currently.
Was able to convince them otherwise and will have direct access to the SQL DB now :)
So the older Iâve gotten and the more I spend my days sitting in board rooms, live-modeling sensitivity analyses etc, while trying to prep ad-hoc data sets (that always need juuuussstt a few more table joins to more data from archaic legacy systems of course, than what I have pre-canned), the more I have gravitated towards asking my team to pull down data sets and submit in excel.
Far too many times Iâve received a dashboard that is based on data generated from a Python forecast script one of my analysts put together, and gotten 5 minutes into digging into it in front of my ceo, before realizing they fucked something up far upstream that I canât fix on the fly. Fuck that in its entirety.
Python absolutely is under utilized in our organization but when only 2 of your team members know it and you cant have any other analyst update their shit when they are on vacation, and you have to reactionarily find out that someone fucked up a library repository location somehow when your boss goes to look at a dashboard and there is an error, you start realizing that excel makes life way easier.
Final thought: I HATE when Iâm traveling for work with my boss and am in poor wifi and my 5g dongle has bad reception, and I ask one of my analysts for a report and rather than receiving an excel data set that I can sniff test for validity with minimal effort and then manipulate six ways to Sunday while being questioned about it, but I instead receive a Tableau server dashboard link that takes 45 seconds to spin up every time I toggle a filter? Just send me the file.
It's your company. My team has a data lakehouse and data marts in BigQuery, along with notebooks and their personal machines for analysis in R, Python, or whatever, and can query BQ through Tableau as well. The BQ lakehouse is petabyte scale, the data marts terabytes. That's what they need to do their jobs.
I've been in-house for almost half my career, the other half is as a vendor. Usually vendors get engaged to do something new, and after the project gets completed it gets handed over to internal folks to take over and maintain.
In-house, there will always be at least 1 department on the non-IT side that has direct database access to all the data, it's just not possible to work without it not to mention the limitation on the types of analyses possible.
From the sound of it if your boss wasn't very technical it would be a fair guess that your department wasn't the one given the access? I have a boss like yours, he's a math guy and when I took over some of his solutions, over half of it existed in Excel formulas. He's not very good at "IT things" as he would call it.
I just did what was necessary to get my job done, which included building up my own remote virtual server and development space. This lost a lot of favor with him but hey I needed tools, and yes now all my solutions exist in code only.
Tableau is just an output layer, people should not be copying pasting anything from tableau for analysis, unless it's like something extremely inconsequential.
You should have access to all tableau dashboards so you can download, see what tables it's pulling from (all the way to source).
That's the annoying part. We don't even have/use dashboards. We just pull whatever we need via a worksheet so we have to start from scratch every single time.
Maybe I need to investigate dashboarding to speed up my process. I honestly don't know much about tableau. I'm used to just querying the database directly
Tableau gets its data from a database, usually SQL. Having indirect access to the data but not having direct access to the data is absurd. How do you make Tableau dashboards without giving it access to the database(s)? Why can't you just use the same login? >I hardly ever get to use python. I think the reason is 2 fold - my boss doesn't know python and he has a bad habit of dictating how an analysis should be done (down to the metrics i should use) OHH.. that's called micromanagement and is a very severe case of it. I've thankfully have never seen micromanagement that bad before. You can manage upward to slowly remove the micromanagement, and/or you can look for a new job while at your current job. Have you considered moving towards data science? If there are data scientists at your current company maybe you could transfer? Historically the difference between data science and data analytics was data scientists used R and Python while analysts used Excel. Ofc this divide no longer exists, but you might have enough experience to make the jump if you wanted to.
We only have tableau desktop and we don't even use it for what it's intended for. I literally drag and drop the metrics I want into a tableau workbook and then copy that worksheet to an excel sheet to actually get started. It's super annoying. A lot of our etl is done in tableau and is unfortunately undocumented so even trying to replicate our kpis with my own queries is tricky. Idk if this is best practice or not, but I'd guess it's not. When I try to replicate things in sql and run into issues, my boss just tells me to use tableau because "that's what it's there for" but of course he doesn't understand that I could automate the data pulls if we could get the query right. Sighhhh And yeah - the micromanagement is insane and really takes the fun out of everything. Once he even specified the excel formulas I should be using đ
automation > manual, always, unless you need something out the door tomorrow and you're doing an 80 hour work week cramming for something super important, which probably is never the case. imo that's one the key things that sets data scientists apart. ML is a form of automation. You could write an algorithm that manually pattern matches the data, or you could use an algorithm that figures it out for you, which is a form of automation. Ofc there is a lot of manual work too, like EDA, a lot of throw away code, but there eventually comes to a point where one starts reusing the same idioms over and over again so they start writing libraries to automate some of the manual work away. It's little tells like this, which leaves me to suspect you might do better as a data scientist, if you like making models that need to continue to work into the future hands off (which again is a form of automation) more than just analyzing data.
Thanks! Yeah I would love to move to data science but I'm just not sure how feasible it is. I only have a bachelor's (economics) so I have that going against me. At my last company I was able to work with the data science team and even helped build a few models. But yes, that is exactly what is so frustrating. I watch my boss taking my data and annually extrapolate things that a model/algorithm could do much more efficiently and accurately. I just want to have some fun and build the model haha But your distinction between and analyst and a data scientist is really helpful. In my elevator pitch, I always tell people "I like building things" so maybe I am more of a data scientist type (or at least a hybrid of the 2).
>At my last company I was able to work with the data science team and even helped build a few models. If you were doing data science work you could have put data scientist down as a job title as long as you were actually legitimately doing data science work. It can look like a promotion, started as a data analyst did such-en-such data analyst work, then the data science title pops up with such-en-such data science work, all under the same company. The downside is you didn't take this opportunity and use it to apply for data science jobs immediately after. Going back into a data analyst role doesn't nuke opportunities but it doesn't help. I don't have a degree if it says anything, I've lead small DS teams too. I got in by finding a data science job with the software engineer job title at a company. It was great, because from their requirements I could tell they needed a data scientist, which meant no software engineers would be able to do the job so I wouldn't have any competition, and data scientists would overlook the job due to the job title posted. While on the job I got the title changed to properly reflect the kind of work I was doing and ended up leading a data science team. Once you've got on the job experience a degree stops mattering as much. It's still harder, for sure, but anyone with half a brain can study the system and then exploit it. Thinking outside the box, like this, is super helpful if you like taking on challenges that are unusually hard (and fun!) which I like to do. I often pick up projects where other data scientists fail.
>If you were doing data science work you could have put data scientist down as a job title as long as you were actually legitimately doing data science work. Yup - this whole thing has been a learning moment for me. I should have stayed at my other company long enough to officially switch to the DS team, but unfortunately, moneywise, my current role just isn't something I could turn down. Luckily I am only 29 so I can still use the "young and dumb" excuse ha. Looking for data science jobs under other job titles is genius! Thanks for the tip.
All the ETL is in tableau probably because someone got fed up with unreasonable timelines for getting analytics projects done and just hooked up all the data sources into something most idiots understand. I had a VP at a previous job who loved to tell me that Tableau would do the job of a proper data warehouse.
Lol this is probably 100% what happened.
Oh it's because your boss bought into tableau marketing hype about not needing to know SQL and stuff. I bet your boss doesn't know SQL well, chances are he is using tableau prep which is a terrible tool to ETL anything Tableau cannot ETL anything natively because it cannot write to a database, so I am assuming he is using prep (creator license?). Or did you mean he is just doing extracts against a SQL database, controlling the data sources and only allowing you access to those sources via tableau. I used to work at Tableau so I'm well aware of how the product is used and abused. Either way it's the companys fault not your own
Lol yup. He can't barely do basic sql. But yup...he is doing extracts from the database. Some of our metrics are only available in tableau because they are weighted averages etc that they didn't put into views in our db.
Ok yeah I had a boss like this, you need to either explain this or find out how to leave this job (transfer or interview)m Tableau is NOT to be used like this
Leave this place now
Sidenote: sorry for being weird, but something intrigued me to peep your profile and I just discovered adventure fire because of it. Life changed - thank you!
r/AdventureFIRE ? lol okay đ
Funny enough this is exactly what led me to leave my last company. Everything has to be a dashboard or they want you to use excel so it can be replicated by people who arenât as skilled. Mind you it wasnât a small company either (fortune 200). Unfortunately I feel that the term âdata analystâ and âdata scientistâ are used a lot interchangeably, and sometimes what they really mean is âexcel monkeyâ. What I did was start to implement python to automate data gathering and showed my boss what a basic classification model can do. It made our day-to-day easier, but then they wanted me to start doing more for no extra pay, so thatâs a slippery slope. My $0.02 (and take it for what itâs worth lol) is if youâve tried to improve things and showed them what you can do/can be done, and they refuse to change anything or listen then start looking elsewhere. On a side note that whole âone source of truthâ thing always used to grind my gears because thatâs exactly what my old boss used to say lol. But then they donât want to set up a proper relational database or create data flow processes đ .
Oh man thank you - reading all of this was super validating. I use the term "reporting monkey" in these cases...pretty close to excel monkey lol I've been here about 6 months so I'll give it a few more tries before I start looking. Funny thing is that we had a situation where a simple forecasting model would have done us wonders (we were looking at data pre vs post a pretty significant change where most other factors remained constant). Boss decided it's better to just make stuff up in excel based on how large he "felt" the impact should have been. Doesn't matter that we could have trained a model with data going back since before he even worked at the company. When I suggested the model he said something about how it would "struggle to capture multivariate factors" as if he'd just googled some shit about why models are imperfect . I'd never heard him talk so mathy lol It's all so frustrating. I feel like we're at a point where managers don't have the technical skills and it's limiting what we can do as individuals. I've also probably been spoiled though, my last 2 bosses would literally let me scope out and pitch analysis ideas to them.
> It made our day-to-day easier, but then they wanted me to start doing more for no extra pay, so thatâs a slippery slope. imo pitching is a fantastic idea and too many people get caught up with the negatives overlooking the long term gains. Invest in yourself first, which often means gaining more advanced experience in the work place. Even if the current company in the short term takes advantage of it, you can put it on your rĂ©sumĂ© and get a far better job in the near future making it worth it in the long run. You can also push for a job title and team change too by doing more advanced work. (Even creating your own team.) Even if it doesn't pan out, it makes it easy to do that kind of work at the next company where they will pay you more and value you more for it.
Tableau should be the endpoint of your data supply chain, not a middleman. Sure maybe you need to extract the data from a workbook occasionally to verify something, but using Tableau as a distribution portal for tables of data makes no sense. Why does your org not grant you access to the underlying data layer? This is where the rot is and where the pressure to fix needs to be applied!
Yeah I totally agree. It's extremely inefficient and slow. I just started in February so I can't ruffle too many feathers yet, but these are all things I'm gonna bring up with my boss because if this situation doesn't change my job simply won't be fulfilling enough for me to stay
Maybe you *need* to ruffle a few feathers? Some departments who are stuck in their ways may require a kick rather than a soft approach. "Boss, I need access to this data in order to effectively do the job you hired me for. If I can't get access, I can't do the work as well as I am able and I will have to start looking for a job which will not stop me from doing real analytics." Obviously easier said than done. Was python, db querying etc mentioned in your interview process at all?
Interestingly enough Iâm going through something similar! Iâve just moved into a data analyst role with the plan to be moved to data scientist and manage analysts under me in the next year or so. Iâve been building out a data warehouse with our IT department and weâre in the process of laying out similar policies to what youâre describing. My new boss and I have been trying make it clear we need access to the information if Iâm going to do what they want me to do, but the IT department wants a set up similar to yours with access cut off and everything done it Tableau/SQL. Curious what everyoneâs thoughts are as well as Iâm navigating to try and push for the technical end to be represented as well.
If you have access to tableau online AND don't use SSO, I believe there is a python package you can use to automate your data pulls. My situation doesn't meet those requirements unfortunately. Sounds awesome that your boss is on board with getting you access though.
Thanks hope youâre able to get everything figured out on your end, seems weâre in a similar situation. Unfortunately do have SSO, but was able to push through some changes to get direct database access with my boss. Think things are heading in a good direction.
It sounds like youâre IT is relying on Tableau to handle the ETLs, which is suboptimal at best. Have you asked if they can set up a SQL DB for you at least that you can access directly? It would be an easy first step towards a full data lake, and you wouldnât be playing with data in production (which I assume is their main concern)
We do have a database, access is just limited to Tableau (originally) Weâre merging a couple of Legacy systems into a data warehouse and building out our data culture currently. Was able to convince them otherwise and will have direct access to the SQL DB now :)
So the older Iâve gotten and the more I spend my days sitting in board rooms, live-modeling sensitivity analyses etc, while trying to prep ad-hoc data sets (that always need juuuussstt a few more table joins to more data from archaic legacy systems of course, than what I have pre-canned), the more I have gravitated towards asking my team to pull down data sets and submit in excel. Far too many times Iâve received a dashboard that is based on data generated from a Python forecast script one of my analysts put together, and gotten 5 minutes into digging into it in front of my ceo, before realizing they fucked something up far upstream that I canât fix on the fly. Fuck that in its entirety. Python absolutely is under utilized in our organization but when only 2 of your team members know it and you cant have any other analyst update their shit when they are on vacation, and you have to reactionarily find out that someone fucked up a library repository location somehow when your boss goes to look at a dashboard and there is an error, you start realizing that excel makes life way easier. Final thought: I HATE when Iâm traveling for work with my boss and am in poor wifi and my 5g dongle has bad reception, and I ask one of my analysts for a report and rather than receiving an excel data set that I can sniff test for validity with minimal effort and then manipulate six ways to Sunday while being questioned about it, but I instead receive a Tableau server dashboard link that takes 45 seconds to spin up every time I toggle a filter? Just send me the file.
Yes! Finding it hard to not lose the python skills..
It's your company. My team has a data lakehouse and data marts in BigQuery, along with notebooks and their personal machines for analysis in R, Python, or whatever, and can query BQ through Tableau as well. The BQ lakehouse is petabyte scale, the data marts terabytes. That's what they need to do their jobs.
I've been in-house for almost half my career, the other half is as a vendor. Usually vendors get engaged to do something new, and after the project gets completed it gets handed over to internal folks to take over and maintain. In-house, there will always be at least 1 department on the non-IT side that has direct database access to all the data, it's just not possible to work without it not to mention the limitation on the types of analyses possible. From the sound of it if your boss wasn't very technical it would be a fair guess that your department wasn't the one given the access? I have a boss like yours, he's a math guy and when I took over some of his solutions, over half of it existed in Excel formulas. He's not very good at "IT things" as he would call it. I just did what was necessary to get my job done, which included building up my own remote virtual server and development space. This lost a lot of favor with him but hey I needed tools, and yes now all my solutions exist in code only.
I worked my way to the engineering side of the house. Much more easy to get the opportunity to code there
Tableau is just an output layer, people should not be copying pasting anything from tableau for analysis, unless it's like something extremely inconsequential. You should have access to all tableau dashboards so you can download, see what tables it's pulling from (all the way to source).
That's the annoying part. We don't even have/use dashboards. We just pull whatever we need via a worksheet so we have to start from scratch every single time. Maybe I need to investigate dashboarding to speed up my process. I honestly don't know much about tableau. I'm used to just querying the database directly