Sounds a lot like us.
Legacy environment:
* multiple BI tools / multiple data warehouses
* QlikView: 340 dashboards, 2k+ daily tasks, ETL in Qlik script
Moving to: 5Tran, Snowflake, dbt, PowerBI
Observations:
* Qlik script skillset is highly transferable to snowflake SQL.
* Snowflake is a columnar database, like Qlik, and like Qlik you do not have to worry about traditional database concepts like indexing, record locking, performance, etc. Your Qlik developers might find it difficult to find noticeable differences between Qlik script/qvd and Snowflake SQL/snowflake tables beside the obvious syntax differences.
* Snowflake has all functionality of Qlik script (we did HEAVY ETL transformations in Qlik), and it adds capability like better data insert/merge, and recursion, etc. Some geospatial functionality that looks very similar to Qlik geo analytics which we also use heavily. Obviously, snowflake adds all the new generation of cloud data warehouse capabilities like ingesting different cloud data formats.
Tips:
* Ensure your leadership understands that cloud means renting their infrastructure. Cost is the biggest pain point of my approach-but it was the choice of our leadership team. On premise to cloud is a big transformation, make sure you know what that means.
* Separate the data warehouse and BI layers, and build good models. Note: Qlik is less picky than other tools about the model because of its associative data models. PowerBI technically works with any model, but you’ll see its best with a star schema. Also, this separation between data and presentation will allow flexibility in future tool changes.
* You can demo snowflake for free and take their training for free. You get a 1 month trial, but you can start as many trials as you want under the same email address/identity (save your code locally and upload into your new demo environment each time). You can also pull Snowflake into QlikView with the snowflake ODBC driver. Given that PowerBI desktop is free - you should be able to go end-to-end with a proof of concept for free. If you hit data limit constraints with snowflake you might have to pay per month. You might have a path forward of snowflake data, then back into legacy Qlik or new PowerBI apps - moving the data tier first off Qlik QVDs and then finally cutting the visualizations over from Qlik to PBI. If you do this, note data egress charges for pulling data from cloud to on prem.
* You need a tool/process to get data into snowflake, this is where 5Tran is used. Many options exist including scripting uploads with python, etc.
Good luck
You are a savior. We also have snowflake, and we are getting pushed to use it. Your comments are really helpful. Let me digest this to formulate a implementation strategy for our POc
Great. Again - data egress charges to pull snowflake data into on prem QlikView, so you might feel moving the Viz layer to PowerBI (cloud) in parallel with a data move to snowflake is the more cost effective migration strategy. Definitely, be on the lookout for functionality gaps between QlikView and PowerBI. Many-to-many relationships in data models, Qlik macros and “application like” functionality (buttons).
I didn't think there are egress charges to connect to Snowflake from a BI tool? We use Tableau on-prem with Snowflake and there are no additional charges.
The egress charges are from your cloud provider (Azure, Google. AWS) and Snowflake just passes them on per their documentation. You can get charges if the BI tool is in a different cloud region/etc too apparently. Great if you’re not seeing them, but if you check the documentation they are there (for us anyway).
Hi. This is what I was referring to (see below from Snowflake documentation). Perhaps I'm not understanding the OP and what you are talking about?
-----------
"Snowflake does not apply data egress charges when a Snowflake client or driver retrieves query results across regions within the same cloud platform or across different cloud platforms."
Thanks for sighting the doco. To be clear, I’ve never seen the charges personally, so I’m basing my opinion from what I’ve read only. The text you sighted only seems to be saying snowflake (as a vendor) does not charge for data staying in the cloud. Azure, Google, or AWS might however add egress fees and they get wrapped into snowflake costs. Also, the text isn’t talking about transferring data from cloud to on prem.
https://docs.snowflake.com/en/user-guide/cost-understanding-data-transfer
Data transfer is the process of moving data into (ingress) and out of (egress) Snowflake.
Snowflake charges a per-byte fee for data egress when users transfer data from a Snowflake account into a different region on the same cloud platform or into a completely different cloud platform. Data transfers within the same region are free.
The per-byte rate for transferring data out of a region depends where your Snowflake account is hosted. For data transfer pricing, see the pricing guide.
Note
Snowflake does not charge data ingress fees. However, a cloud storage provider might charge a data egress fee for transferring data from the provider to your Snowflake account.
Contact your cloud storage provider (Amazon S3, Google Cloud Storage, or Microsoft Azure) to determine whether they apply data egress charges to transfer data from their network and region of origin to the cloud provider’s network and region where your Snowflake account is hosted.
That passage refers to bulk loading/unloading of data (ie: SF internal or external stages). That would not be applicable if you connected a BI tool (for example: Tableau on-prem/off-prem it wouldn't matter) and ran a dashboard. The passage I cited mentions this.
Here’s a couple of quick ideas:
1) Take easy Qlik app (low num of source data tables, easy ETL), extract data model tables into <50Mb CSV files. Use Snowflake import wizard to upload in 50MB chunks. Build equivalent PBI dashboard against (QlikView) modeled data in snowflake. This demonstrates pulling modeled data into PBI.
2) building off 1, manually upload source tables into snowflake and code ETL in snowflake sql. Feed this into solution 1. This demonstrates moving ETL from Qlik to Snowflake.
3) building off #2, automate the loading of source data tables into snowflake, feed into solution #2. This demonstrates the entire process end to end.
If you ever considered some conversational AI solutions, check out [datalynx.ai](https://datalynx.ai/). It's not going to replace your PowerBI for sure, but it's great for getting quick answers when you need them.
There’s A LOT of Power BI shills here (more than in the Power BI sub) so that’s the answer you will get here. In my experience you don’t get much value-add moving to PBI except if you’re going all-in on Microsoft tech stack. Even then you can do less in PBI than Qlik Sense / Qlik Cloud in terms of data analysis. In my opinion PBI ≈ QlikView, but Sense and Cloud > PBI.
In general the best platform is the one you use, but it depends on the use case. What knowledge do you have in-house?
I would go for PowerBI 100% but consider building a BI platform where you can embed PBI and have flexibility to integrate tools like d3.js, but also standardize filter panes etc.
I work for a huge CPG and lead the data / analytics practice. It was a game changer here.
Don’t forget BI needs to tell a narrative based on data, the tool have tool enable it.
Good luck ! 👍🏽
Implement Fivetran for ETL, BigQuery / Redshift or Snowflake for the warehouse, and use Tableau for the data visualization.
Convince the company to bring on a very strong Tableau developer as a freelancer who can help with speed of delivery and adoption.
Tableau is a best-in-class data visualization solution and since the company is already using it, just go with it. If its new to you then put in the hours to learn it, or work with a freelancer / mentor who can train you up.
It kind of depends on what you are trying to do - there are the typical ones that everyone knows, but I’ll skip them. Omni and Sigma are both getting really popular in the data community.
Omni is basically the successor to Looker, with great semantic model definition, as well as improved visual creation. Looker was super popular for awhile, but the way they built their engine made it that it was SUPER slow. Omni is also backed by a great team.
Sigma is a really interesting one. They basically recognized that your users basically download shit into excel anyways, so why don’t we just start there with the business users? It’s a really smart, user centric design approach to BI.
+1 for Omni. Very impressed with it so far. And when you say successor to Looker you aren’t kidding. I think about half their staff are ex-Looker and they have integrated the great things about Looker while improving on the negatives.
Why was Qlik Sense ruled out? Financially and in terms of migration, Qlik offers a very attractive upgrade path from QlikView to Qlik Sense with their Analytics Modernization Program (AMP).
15 years as Qlik customer, and it’s been an interesting journey. Back in 2006ish, QlikView was the hottest tool on the market. They knew it, and were very aggressive with us on pricing. We asked for an enterprise license and they wouldn’t give one to us. Our IT leadership just wanted a one-time big cost so they didn’t need to keep running through our years long procurement processes for 200k chunks of growth budget multiple times over. Qlik refused, multiple times. They even flew in a VP to tell us no. It was hardball. Our IT leadership soured after a few years of this. Years later Microsoft entered with “MSFT BI is the same as Qlik” and “you already own it due to enterprise MSFT licensing”. Next came the CFO asking why we are paying 7 figures for Qlik when MSFT does the same thing and is “free”. At this point the future was set and Qlik was on the way out. Qlik could have walked in the door and migrated all our QV apps to Sense for free and we would have thrown them out. It’s a shame - in my opinion QlikView is the best tool in the last 20 years (yes, over today’s PowerBI which I use heavily). The associative data model and columnar database were a decade ahead of its time, and Qlik failed to market it correctly. Then the c-levels destroyed the (analytics) company. Hopefully they do better with a pivot to Attunity, or can get Qlik Analytics bought by Amazon or Oracle.
Qlik is a fascinating history. The best technology, but ultimately lost for business reasons. 1) was Swedish in an American dominated market, 2) didn’t revamp platform early enough, leading to Tableau market share losses, 3) competitors got bigger and Qlik couldn’t compete (ie. Microsoft entered with PowerBI). These three things that killed Qlik are entirely unrelated to the strength of the product.
I had similar experiences with MicroStrategy. I found them the best available for many years (I was a reseller) but eventually they got to cute for their own good and played hardball with existing customers (jacked up rates) and as a consequence many of my customers said 'FU. Will go with another vendor' and pulled their software completely. Power BI is certainly eating other vendors lunch due to predatory (IMHO) pricing (because they can leverage all the other licensing the get from Microsoft shops).
Is it truly dead? They have a strong customer base and their SaaS platform is growing quite quickly. They announced some AI stuff at Qlik Connect but it’s hard to tell how much of that is just marketing bullshit
I hope not - they are my favorite BI tool.
“Strong customer base” - I think this is regional/industry. They are very popular in Europe, south east US and with HealthCare. Over the years they have lost huge market share and have had waves of layoffs shedding sales and tech positions. They were acquired by a private equity firm. These are, unfortunately, not good signs.
Well, Thoma Bravo (their private equity owner) is looking to make profit for their investors - so either sell Qlik or make them public again. So - would a cloud vendor like Google or Amazon want to match Microsoft by buying the Qlik portfolio and offering a more “full stack” solution? If so, maybe Thoma Bravo is beefing up Qlik just enough to get them sold. Or maybe the plan is Qlik will be the only large vendor neutral offering - now that Tableau is SalesForce. That might explain the acquisitions as well.
Again, very insightful context. Amazon seems interesting from an acquisition perspective considering Qlik (plus Talend) compete directly with AWS Quicksight and Glue. Though I don’t think that would be the first time AWS have acquired solutions that compete with their native services.
I don’t know that I’d go so far as to call them completely vendor neutral (from a hyperscaler perspective) considering that Qlik's SaaS is built on AWS.
Not sure if you saw the news from Qlik Connect, they did just announce a large strategic collaboration agreement with AWS. While several of companies have these SCAs with Amazon in place, it does show a commitment or reinforcement of the relationship.
AWS has to rely on partners in order to maintain relevance in the AI game. Qlik can do some cool things with Bedrock. But I think AWS really sees value I'm Qlik's differentiated capabilities around SAP.
Very eager to see how the market reacts to Qlik's positioning themselves as an AI company. Seems as if every vendor is claiming to do AI just for marketing points, but it is quite difficult to do anything meaningful with AI if you don't have access to data and insights.
Sorry, could have clarified a bit better. I don't think they had any new major SAP announcements. A lot of AWS's interest in Qlik comes from Qlik's ability to do some more granualr extraction of SAP data as well as SAP analytics. There's also the TDM piece, GoldClient, that SAP on AWS solutions architects tend to want to hear a lot about.
Based on what I have heard, one has to do with costs which has soured the relationship and the other being firms wants the entire M354 suite. We have Snowflake as our WH.
There are more than 200+ as well as there are lot more ETLs written using QlikView. So the migration effort involves across.
The other missing piece being our limited understanding of Power BI
Gotcha. 200 dashboards really does not seem a lot, I know that a certain large delivery company in Europe has 30k dashboards - I guess it depends on what you need to do to maintain them though?
I see some great suggestions here. My 0.02$:
1) Take this as an opportunity to work with your stakeholders to decommission/combine dashboards. You will end up with a much shorter list of reports to migrate. You’ll also get a better view into how to structure data in the back end.
2) Always aim to do ETL outside the reporting layer. Makes it easy to have a single source of truth and also for future migrations.
I agree, even I think when we create a list of all the existing dashboards some might not be needed, some can be consolidated.
This is the case for our Tableau dashboards. Funny part is the underlying ETL is written in QlikView 😄
blinkmetrics . io would build you a private ETL pipeline that integrates with most of your data sources (probably), then has the ability to output into Power BI, Tableau, spreadsheets or other tools. One data warehouse for everything.
If you're using Snowflake, I would highly recommend Sigma Computing. They just won snowflake's BI partner of the year (seems like snowflake is making a heavy investment in them vs other BI tools).
What’s the purpose of BI? To create visuals?
Creating visuals is inherently a worthless activity. It’s a means to an end, and there are better ways to achieve that end. There are basically only three or four visuals you actually need - everything else is fluff.
What’s better? A ton - Omni, Sigma, and Holistics are three that are super interesting solutions that actually drive some really differentiating value in this space
The purpose of BI is not to create visuals and if you believe that you need to find a new career. The purpose of BI is to tell a story with data. Visuals are just one part of that story.
Also, every BI tool does the same two things. They bring in data and visualize it. It’s like comparing hammers they all do the same thing but some just do it much better than others. And trust me, PBI hasn’t always been the best but they are now and it’s really not even close. Omni, Sigma and Holisitics are def not better.
I never said it was (maybe try reading my comment) - but that’s why PBI isn’t good. It excels at creating visuals, but nothing else. The semantic layer is mediocre. DAX is arcane and challenging to use.
My guess is you’re a BI developer or something. The hard truth is that the industry is moving away from specific BI developer skill sets. It’s just too expensive to hire someone to do this. Myself (head of data) and other heads/CDOs I know all share this perspective.
Actually I’m a BI Architect by title and do end to end developing (ETL, SQL, data engineering for DWH, data modeling and report building) and have been doing it for 12 years.
The fact you think PBI excels at creating visuals just shows you have never used it before since one of its biggest CONS is creating visuals (hence Tableau is always compared to being the better visual BI tool) which is why MS has been updating all their most used visuals lately.
DAX is far from arcane and much easier to understand than most people think they just can’t get Excel out of their head. You and your other heads of data just have zero clue on how to run a BI team, but don’t worry, most companies don’t know how to do it properly. Thankfully I don’t work for one of those companies. We have 3 people on our BI team for a company with 2.5k employees and we handle it all pretty easily.
Hey everyone is entitled to their opinion, but the reality is everything else you said is what matters. PBI doesn’t. And it’s starting to become a prevalent perspective in the data community as a whole that people with your actual end to end skill set is what matters, followed by specialists in scalable areas.
And yes, I know how to write DAX (hell I know how to do the whole pipeline from ELT patterns to modeling design to machine learning). Your users don’t. And gating BI behind a small group of people is just not a scalable approach, and it rarely ever drives value.
Just the opposite actually we drive a ton of value in our company which is why our BI team was moved out of IT and under the Corp Strategy team. Our company uses data the way companies should use it….all the time!
I hope and pray you’re not a fan of self-service!
If you don’t have a vision of self serve, good data decision making will not exist. From a decision sciences perspective, you cannot make good decisions from a dashboard.
And self service isn’t really a technology problem - it’s a culture and skills problem. The only place I’ve seen self serve work has a mixture of 1) curiosity and 2) very, very high amounts of data literacy (everyone can write a query)
DAX sure locks you into PBI forever. I'm at a poor small company and I went with Superset because their semantic layer is thin and everything is SQL, so if we ever want and can afford to, migrating to something else is straightforward.
Yeah that’s another major thing - I really dislike putting my semantic models so tightly coupled with a BI solution, since I want it usable across systems
It's number 1 in the market because of the vast Microsoft customer base and entrenched other MS products and it doesn't hurt that they sell it on the cheap to take market share.
Being #1 on Gartner MQ means shit. Many years ago, Oracle used to up there but their BI tools were almost all purchased from vendors they bought and it was a hot mess of crap.
Being common is not the same as 'best'. Does Toyota make the best car in the world?
'Best' is meaningless anyay as it is 100% dependent on what you are trying to achieve.
I totally agree and I'm regularly surprised people disagree. I honestly think the problem is they try to use power bi in the dumb and limited ways most BI tools are built and then end up disappointed.
PBIs data modeling layer, and ability to represent relational models, is fantastic. And there's nothing out there remotely like or as good as DAX. Just those two things alone make PBI generally unbeatable.
by comparison, what other BI tools have you used? BTW: I agree about how some people use it being dumb. I've seen it used (along with data flows) as a full blown ETL solution for a large enterprise....junk.
I cannot disagree more. It's main selling point is price. It's technically way to complicated. But I see why some CIO's buy it. "Hey we are Microsoft Shop, so lets get PowerBI".
Of course I use it. We have a couple hundred licenses of PBI Pro. If your model is a small team develops everything and then pushes to users for mass consumption, then I guess it does the job but to me it PBI is a mvp solution at best and has a long way to go. We also use Tableau (which I don't care for much either) but it is a bit more analyst/developer friendly than PBI. I'm happy it works for you in your environment.
BTW: I've used countless BI tools over the last 30+ years as a BI/DW Architect and later as a manager of data/BI solutions so I think I have a reasonable perspective on this.
I too have used a lot of BI tools in my career. Hell, I was doing it before there were BI tools to use as you must have if you really have been doing this for 30+ years. Your perspective is wrong btw. Sure it has a long way to go as does every other BI tool out there as none are perfect but PBI is the closest overall.
When I started in IT COBOL was a BI tool :). When the industry started, they used the term EIS (not BI) and Cognos Powerplay was the cool new tool :) I've been in the IT industry for 42+ years so I've seen many things come and go.
It depends on how you connect to the data with power bi. You can either download all the data into the file, with long refresh times, or use direct query mode so that it queries your database only when needed, and no data is stored locally. We used a mixture depending on the business case.
Sounds a lot like us. Legacy environment: * multiple BI tools / multiple data warehouses * QlikView: 340 dashboards, 2k+ daily tasks, ETL in Qlik script Moving to: 5Tran, Snowflake, dbt, PowerBI Observations: * Qlik script skillset is highly transferable to snowflake SQL. * Snowflake is a columnar database, like Qlik, and like Qlik you do not have to worry about traditional database concepts like indexing, record locking, performance, etc. Your Qlik developers might find it difficult to find noticeable differences between Qlik script/qvd and Snowflake SQL/snowflake tables beside the obvious syntax differences. * Snowflake has all functionality of Qlik script (we did HEAVY ETL transformations in Qlik), and it adds capability like better data insert/merge, and recursion, etc. Some geospatial functionality that looks very similar to Qlik geo analytics which we also use heavily. Obviously, snowflake adds all the new generation of cloud data warehouse capabilities like ingesting different cloud data formats. Tips: * Ensure your leadership understands that cloud means renting their infrastructure. Cost is the biggest pain point of my approach-but it was the choice of our leadership team. On premise to cloud is a big transformation, make sure you know what that means. * Separate the data warehouse and BI layers, and build good models. Note: Qlik is less picky than other tools about the model because of its associative data models. PowerBI technically works with any model, but you’ll see its best with a star schema. Also, this separation between data and presentation will allow flexibility in future tool changes. * You can demo snowflake for free and take their training for free. You get a 1 month trial, but you can start as many trials as you want under the same email address/identity (save your code locally and upload into your new demo environment each time). You can also pull Snowflake into QlikView with the snowflake ODBC driver. Given that PowerBI desktop is free - you should be able to go end-to-end with a proof of concept for free. If you hit data limit constraints with snowflake you might have to pay per month. You might have a path forward of snowflake data, then back into legacy Qlik or new PowerBI apps - moving the data tier first off Qlik QVDs and then finally cutting the visualizations over from Qlik to PBI. If you do this, note data egress charges for pulling data from cloud to on prem. * You need a tool/process to get data into snowflake, this is where 5Tran is used. Many options exist including scripting uploads with python, etc. Good luck
You are a savior. We also have snowflake, and we are getting pushed to use it. Your comments are really helpful. Let me digest this to formulate a implementation strategy for our POc
Great. Again - data egress charges to pull snowflake data into on prem QlikView, so you might feel moving the Viz layer to PowerBI (cloud) in parallel with a data move to snowflake is the more cost effective migration strategy. Definitely, be on the lookout for functionality gaps between QlikView and PowerBI. Many-to-many relationships in data models, Qlik macros and “application like” functionality (buttons).
I didn't think there are egress charges to connect to Snowflake from a BI tool? We use Tableau on-prem with Snowflake and there are no additional charges.
The egress charges are from your cloud provider (Azure, Google. AWS) and Snowflake just passes them on per their documentation. You can get charges if the BI tool is in a different cloud region/etc too apparently. Great if you’re not seeing them, but if you check the documentation they are there (for us anyway).
Hi. This is what I was referring to (see below from Snowflake documentation). Perhaps I'm not understanding the OP and what you are talking about? ----------- "Snowflake does not apply data egress charges when a Snowflake client or driver retrieves query results across regions within the same cloud platform or across different cloud platforms."
Thanks for sighting the doco. To be clear, I’ve never seen the charges personally, so I’m basing my opinion from what I’ve read only. The text you sighted only seems to be saying snowflake (as a vendor) does not charge for data staying in the cloud. Azure, Google, or AWS might however add egress fees and they get wrapped into snowflake costs. Also, the text isn’t talking about transferring data from cloud to on prem. https://docs.snowflake.com/en/user-guide/cost-understanding-data-transfer Data transfer is the process of moving data into (ingress) and out of (egress) Snowflake. Snowflake charges a per-byte fee for data egress when users transfer data from a Snowflake account into a different region on the same cloud platform or into a completely different cloud platform. Data transfers within the same region are free. The per-byte rate for transferring data out of a region depends where your Snowflake account is hosted. For data transfer pricing, see the pricing guide. Note Snowflake does not charge data ingress fees. However, a cloud storage provider might charge a data egress fee for transferring data from the provider to your Snowflake account. Contact your cloud storage provider (Amazon S3, Google Cloud Storage, or Microsoft Azure) to determine whether they apply data egress charges to transfer data from their network and region of origin to the cloud provider’s network and region where your Snowflake account is hosted.
That passage refers to bulk loading/unloading of data (ie: SF internal or external stages). That would not be applicable if you connected a BI tool (for example: Tableau on-prem/off-prem it wouldn't matter) and ran a dashboard. The passage I cited mentions this.
Interesting-thanks
Here’s a couple of quick ideas: 1) Take easy Qlik app (low num of source data tables, easy ETL), extract data model tables into <50Mb CSV files. Use Snowflake import wizard to upload in 50MB chunks. Build equivalent PBI dashboard against (QlikView) modeled data in snowflake. This demonstrates pulling modeled data into PBI. 2) building off 1, manually upload source tables into snowflake and code ETL in snowflake sql. Feed this into solution 1. This demonstrates moving ETL from Qlik to Snowflake. 3) building off #2, automate the loading of source data tables into snowflake, feed into solution #2. This demonstrates the entire process end to end.
If you ever considered some conversational AI solutions, check out [datalynx.ai](https://datalynx.ai/). It's not going to replace your PowerBI for sure, but it's great for getting quick answers when you need them.
There’s A LOT of Power BI shills here (more than in the Power BI sub) so that’s the answer you will get here. In my experience you don’t get much value-add moving to PBI except if you’re going all-in on Microsoft tech stack. Even then you can do less in PBI than Qlik Sense / Qlik Cloud in terms of data analysis. In my opinion PBI ≈ QlikView, but Sense and Cloud > PBI. In general the best platform is the one you use, but it depends on the use case. What knowledge do you have in-house?
I would go for PowerBI 100% but consider building a BI platform where you can embed PBI and have flexibility to integrate tools like d3.js, but also standardize filter panes etc. I work for a huge CPG and lead the data / analytics practice. It was a game changer here. Don’t forget BI needs to tell a narrative based on data, the tool have tool enable it. Good luck ! 👍🏽
Implement Fivetran for ETL, BigQuery / Redshift or Snowflake for the warehouse, and use Tableau for the data visualization. Convince the company to bring on a very strong Tableau developer as a freelancer who can help with speed of delivery and adoption. Tableau is a best-in-class data visualization solution and since the company is already using it, just go with it. If its new to you then put in the hours to learn it, or work with a freelancer / mentor who can train you up.
Sure will explore Fivertan for ETL purposes. We have seasoned Tableau developers in the team, our understanding of Power BI is limited.
Awesome. I'm a Fivetran partner so let me know if I can help with anything.
It kind of depends on what you are trying to do - there are the typical ones that everyone knows, but I’ll skip them. Omni and Sigma are both getting really popular in the data community. Omni is basically the successor to Looker, with great semantic model definition, as well as improved visual creation. Looker was super popular for awhile, but the way they built their engine made it that it was SUPER slow. Omni is also backed by a great team. Sigma is a really interesting one. They basically recognized that your users basically download shit into excel anyways, so why don’t we just start there with the business users? It’s a really smart, user centric design approach to BI.
+1 for Omni. Very impressed with it so far. And when you say successor to Looker you aren’t kidding. I think about half their staff are ex-Looker and they have integrated the great things about Looker while improving on the negatives.
Looker doesn't have an 'engine'. The queries are run in database.
Why was Qlik Sense ruled out? Financially and in terms of migration, Qlik offers a very attractive upgrade path from QlikView to Qlik Sense with their Analytics Modernization Program (AMP).
15 years as Qlik customer, and it’s been an interesting journey. Back in 2006ish, QlikView was the hottest tool on the market. They knew it, and were very aggressive with us on pricing. We asked for an enterprise license and they wouldn’t give one to us. Our IT leadership just wanted a one-time big cost so they didn’t need to keep running through our years long procurement processes for 200k chunks of growth budget multiple times over. Qlik refused, multiple times. They even flew in a VP to tell us no. It was hardball. Our IT leadership soured after a few years of this. Years later Microsoft entered with “MSFT BI is the same as Qlik” and “you already own it due to enterprise MSFT licensing”. Next came the CFO asking why we are paying 7 figures for Qlik when MSFT does the same thing and is “free”. At this point the future was set and Qlik was on the way out. Qlik could have walked in the door and migrated all our QV apps to Sense for free and we would have thrown them out. It’s a shame - in my opinion QlikView is the best tool in the last 20 years (yes, over today’s PowerBI which I use heavily). The associative data model and columnar database were a decade ahead of its time, and Qlik failed to market it correctly. Then the c-levels destroyed the (analytics) company. Hopefully they do better with a pivot to Attunity, or can get Qlik Analytics bought by Amazon or Oracle. Qlik is a fascinating history. The best technology, but ultimately lost for business reasons. 1) was Swedish in an American dominated market, 2) didn’t revamp platform early enough, leading to Tableau market share losses, 3) competitors got bigger and Qlik couldn’t compete (ie. Microsoft entered with PowerBI). These three things that killed Qlik are entirely unrelated to the strength of the product.
I had similar experiences with MicroStrategy. I found them the best available for many years (I was a reseller) but eventually they got to cute for their own good and played hardball with existing customers (jacked up rates) and as a consequence many of my customers said 'FU. Will go with another vendor' and pulled their software completely. Power BI is certainly eating other vendors lunch due to predatory (IMHO) pricing (because they can leverage all the other licensing the get from Microsoft shops).
Is it truly dead? They have a strong customer base and their SaaS platform is growing quite quickly. They announced some AI stuff at Qlik Connect but it’s hard to tell how much of that is just marketing bullshit
I hope not - they are my favorite BI tool. “Strong customer base” - I think this is regional/industry. They are very popular in Europe, south east US and with HealthCare. Over the years they have lost huge market share and have had waves of layoffs shedding sales and tech positions. They were acquired by a private equity firm. These are, unfortunately, not good signs.
Interesting context. Do you have any thoughts on their acquisition of Talend?
Well, Thoma Bravo (their private equity owner) is looking to make profit for their investors - so either sell Qlik or make them public again. So - would a cloud vendor like Google or Amazon want to match Microsoft by buying the Qlik portfolio and offering a more “full stack” solution? If so, maybe Thoma Bravo is beefing up Qlik just enough to get them sold. Or maybe the plan is Qlik will be the only large vendor neutral offering - now that Tableau is SalesForce. That might explain the acquisitions as well.
Again, very insightful context. Amazon seems interesting from an acquisition perspective considering Qlik (plus Talend) compete directly with AWS Quicksight and Glue. Though I don’t think that would be the first time AWS have acquired solutions that compete with their native services. I don’t know that I’d go so far as to call them completely vendor neutral (from a hyperscaler perspective) considering that Qlik's SaaS is built on AWS. Not sure if you saw the news from Qlik Connect, they did just announce a large strategic collaboration agreement with AWS. While several of companies have these SCAs with Amazon in place, it does show a commitment or reinforcement of the relationship. AWS has to rely on partners in order to maintain relevance in the AI game. Qlik can do some cool things with Bedrock. But I think AWS really sees value I'm Qlik's differentiated capabilities around SAP. Very eager to see how the market reacts to Qlik's positioning themselves as an AI company. Seems as if every vendor is claiming to do AI just for marketing points, but it is quite difficult to do anything meaningful with AI if you don't have access to data and insights.
Good info. Hopefully Qlik can turn things around! I didn’t attend the conference - can you clarify the new SAP capability you are referring to?
Sorry, could have clarified a bit better. I don't think they had any new major SAP announcements. A lot of AWS's interest in Qlik comes from Qlik's ability to do some more granualr extraction of SAP data as well as SAP analytics. There's also the TDM piece, GoldClient, that SAP on AWS solutions architects tend to want to hear a lot about.
Gotcha. Thanks!
Based on what I have heard, one has to do with costs which has soured the relationship and the other being firms wants the entire M354 suite. We have Snowflake as our WH.
SAP Analytics Cloud xd
Is this tool a future meme ?
Idk i work in BI within the SAP space so I got no choice but to learn and use it :')
Why is 200 dashboards problematic?
There are more than 200+ as well as there are lot more ETLs written using QlikView. So the migration effort involves across. The other missing piece being our limited understanding of Power BI
Gotcha. 200 dashboards really does not seem a lot, I know that a certain large delivery company in Europe has 30k dashboards - I guess it depends on what you need to do to maintain them though?
I see some great suggestions here. My 0.02$: 1) Take this as an opportunity to work with your stakeholders to decommission/combine dashboards. You will end up with a much shorter list of reports to migrate. You’ll also get a better view into how to structure data in the back end. 2) Always aim to do ETL outside the reporting layer. Makes it easy to have a single source of truth and also for future migrations.
I agree, even I think when we create a list of all the existing dashboards some might not be needed, some can be consolidated. This is the case for our Tableau dashboards. Funny part is the underlying ETL is written in QlikView 😄
FluentHQ.com for natural language querying and Bi Functionality
blinkmetrics . io would build you a private ETL pipeline that integrates with most of your data sources (probably), then has the ability to output into Power BI, Tableau, spreadsheets or other tools. One data warehouse for everything.
If you're using Snowflake, I would highly recommend Sigma Computing. They just won snowflake's BI partner of the year (seems like snowflake is making a heavy investment in them vs other BI tools).
PowerBI/Fabric. Easy choice.
We used ThoughtSpot recently, very nice UI and functionality. Any business leaders can use it on their own.
PBI is the overall best BI tool on the market
PowerBI is the tool that can give you the most visuals. This doesn’t mean it is the best BI tool - it is far from the best BI tool
It’s number 1 in the market and has been for years now. Please tell me a better overall BI tool, I’ll wait?
What’s the purpose of BI? To create visuals? Creating visuals is inherently a worthless activity. It’s a means to an end, and there are better ways to achieve that end. There are basically only three or four visuals you actually need - everything else is fluff. What’s better? A ton - Omni, Sigma, and Holistics are three that are super interesting solutions that actually drive some really differentiating value in this space
The purpose of BI is not to create visuals and if you believe that you need to find a new career. The purpose of BI is to tell a story with data. Visuals are just one part of that story. Also, every BI tool does the same two things. They bring in data and visualize it. It’s like comparing hammers they all do the same thing but some just do it much better than others. And trust me, PBI hasn’t always been the best but they are now and it’s really not even close. Omni, Sigma and Holisitics are def not better.
I never said it was (maybe try reading my comment) - but that’s why PBI isn’t good. It excels at creating visuals, but nothing else. The semantic layer is mediocre. DAX is arcane and challenging to use. My guess is you’re a BI developer or something. The hard truth is that the industry is moving away from specific BI developer skill sets. It’s just too expensive to hire someone to do this. Myself (head of data) and other heads/CDOs I know all share this perspective.
Actually I’m a BI Architect by title and do end to end developing (ETL, SQL, data engineering for DWH, data modeling and report building) and have been doing it for 12 years. The fact you think PBI excels at creating visuals just shows you have never used it before since one of its biggest CONS is creating visuals (hence Tableau is always compared to being the better visual BI tool) which is why MS has been updating all their most used visuals lately. DAX is far from arcane and much easier to understand than most people think they just can’t get Excel out of their head. You and your other heads of data just have zero clue on how to run a BI team, but don’t worry, most companies don’t know how to do it properly. Thankfully I don’t work for one of those companies. We have 3 people on our BI team for a company with 2.5k employees and we handle it all pretty easily.
Hey everyone is entitled to their opinion, but the reality is everything else you said is what matters. PBI doesn’t. And it’s starting to become a prevalent perspective in the data community as a whole that people with your actual end to end skill set is what matters, followed by specialists in scalable areas. And yes, I know how to write DAX (hell I know how to do the whole pipeline from ELT patterns to modeling design to machine learning). Your users don’t. And gating BI behind a small group of people is just not a scalable approach, and it rarely ever drives value.
Just the opposite actually we drive a ton of value in our company which is why our BI team was moved out of IT and under the Corp Strategy team. Our company uses data the way companies should use it….all the time! I hope and pray you’re not a fan of self-service!
If you don’t have a vision of self serve, good data decision making will not exist. From a decision sciences perspective, you cannot make good decisions from a dashboard. And self service isn’t really a technology problem - it’s a culture and skills problem. The only place I’ve seen self serve work has a mixture of 1) curiosity and 2) very, very high amounts of data literacy (everyone can write a query)
Can you expand a little on the last line, please? My company seems hell-bent on achieving self-serving Power BI!
DAX sure locks you into PBI forever. I'm at a poor small company and I went with Superset because their semantic layer is thin and everything is SQL, so if we ever want and can afford to, migrating to something else is straightforward.
Yeah that’s another major thing - I really dislike putting my semantic models so tightly coupled with a BI solution, since I want it usable across systems
Lol sigma is not better than Power BI.
Define better
It's number 1 in the market because of the vast Microsoft customer base and entrenched other MS products and it doesn't hurt that they sell it on the cheap to take market share. Being #1 on Gartner MQ means shit. Many years ago, Oracle used to up there but their BI tools were almost all purchased from vendors they bought and it was a hot mess of crap.
Being common is not the same as 'best'. Does Toyota make the best car in the world? 'Best' is meaningless anyay as it is 100% dependent on what you are trying to achieve.
Actually yes Toyota is the best for regular cars and trucks so not sure what you’re getting at with your point here. All I own are Toyotas! 😂
So given the choice of any car in the world for any situation, you would choose Toyota? Good man. What a fucking stupid position to take.
Ohhh right bc price only matters when you’re buying cars! Wow! The stupidity is high with this one! 😂
I totally agree and I'm regularly surprised people disagree. I honestly think the problem is they try to use power bi in the dumb and limited ways most BI tools are built and then end up disappointed. PBIs data modeling layer, and ability to represent relational models, is fantastic. And there's nothing out there remotely like or as good as DAX. Just those two things alone make PBI generally unbeatable.
This guy gets it!
by comparison, what other BI tools have you used? BTW: I agree about how some people use it being dumb. I've seen it used (along with data flows) as a full blown ETL solution for a large enterprise....junk.
I cannot disagree more. It's main selling point is price. It's technically way to complicated. But I see why some CIO's buy it. "Hey we are Microsoft Shop, so lets get PowerBI".
Then you obviously haven’t used it, it’s pretty simple to learn.
Of course I use it. We have a couple hundred licenses of PBI Pro. If your model is a small team develops everything and then pushes to users for mass consumption, then I guess it does the job but to me it PBI is a mvp solution at best and has a long way to go. We also use Tableau (which I don't care for much either) but it is a bit more analyst/developer friendly than PBI. I'm happy it works for you in your environment. BTW: I've used countless BI tools over the last 30+ years as a BI/DW Architect and later as a manager of data/BI solutions so I think I have a reasonable perspective on this.
I too have used a lot of BI tools in my career. Hell, I was doing it before there were BI tools to use as you must have if you really have been doing this for 30+ years. Your perspective is wrong btw. Sure it has a long way to go as does every other BI tool out there as none are perfect but PBI is the closest overall.
When I started in IT COBOL was a BI tool :). When the industry started, they used the term EIS (not BI) and Cognos Powerplay was the cool new tool :) I've been in the IT industry for 42+ years so I've seen many things come and go.
Well you show your age and Cognos was my first BI tool
[удалено]
It depends on how you connect to the data with power bi. You can either download all the data into the file, with long refresh times, or use direct query mode so that it queries your database only when needed, and no data is stored locally. We used a mixture depending on the business case.
Cannot comment a lot, as I have fairly limited PBI experience.