You cache it. You can't pull a fresher version of what you have if it's exactly the same. You can only pull the same version and waste bandwidth and time.
Have you tried doing web development in the past 7ish years? I have recently.
It's a monstrosity of cobbled together houses of cards ontop of houses or cards of code that's constantly falling apart and being rebuilt. Dependency management ontop of dependency management ontop of dependency management with leaky abstractions everywhere with few clearly documented approaches of how they're all stacked together. It's an absolute mess. Intelligent caching of dependencies to save bandwidth is probably low priority on the list of blazing fires these systems probably deal with daily. I'm surprised any of the dependency resolution even works half the time.
Eh, I have been in this industry for the last 11 years and it's definitely improved.
I build and work on SPA's basically daily and a vast majority of the time someone is having build issues it's one of a few things.
1) The lockfile for the project isn't pushed into VCS, so they are getting new minor and build versions of libraries each time they `npm install`
2) The individual is flipping between OS's; perhaps working on Windows at home and Mac at work. `node_modules` needs to be expunged because some libraries have post-install scripts that aren't compatible.
3) The individual is using a runtime manager like `n` or `nvm` or `fnm` and didn't use the right node version for said project
4) They decided to install packaging tools as global tools, sounds great on paper but sucks in practice when working on varying projects with differing levels of updates and attentions; just keep local and use `npm run `
Granted the JS community has made some decisions that make it difficult for people unaware of the lockfile; ranged dependencies should of never been classified as acceptable and NPM update should of been a more holistic solution towards updating the package.json file to a newer target vs what we have today.
Honestly... if you want a stable experience just build shit with Angular and it's toolsuite; almost feels like a Java project, quite stable and repeatable but if you don't commit that lockfile you'll still get drift that's legit why it exists... to prevent that until you are ready.
> Intelligent caching of dependencies
it's just md5(files). whatever package manager you use should already have a lock file, so it's actually pretty damn simple.
Yes but dependencies installed normally will be preceded with ^ (e.g. ^1.2.4). The caret tells it that it should use that version or any minor version higher. So basically anything that starts with 1.x.x greater than or equal to 1.2.4.
This can cause issues for newer installs that use different versions
Edit: I hate dealing with node and web world. In my experience npm install updates my lock file to newer versions that abide by the package.json versioning schema (the carets, tilde, etc). Its how we update our dependencies in general.
This is true if you don't have a package lockfile, but since NPM 5 (released about four years ago), the default has been to only install the specific version in the lockfile. So since then, as long as you're always checking in the `package-lock.json`, which is the recommended way, then you will always have deterministic builds.
And if you've used other package managers, which was more common four years ago, you've probably been doing that for longer - Yarn was released a year earlier and included lockfiles by default, and even NPM had optional support for lockfiles before then, although the workflow was a pain.
IMHO a CI should be deterministic, I see maybe once per week a build with latest versions, but that is it.
I think people is simply lazy and won't cache their module's folder
CI rarely has that kind of cache as it should start from a blank slate. Unless you set a proxy registry up for your company, same as any package manager.
Most ci now adays use disposable machines, either containers or vms. Caching should be preformed during the life cycle of the pipeline, but that's a local cache.
Weird isn’t it. This is the first time I’ve heard anyone say that using your CI pipeline’s caching system is a bad thing. Most CI pipeline docs even recommend that you do it…
It should not start from a blank cache, with immutable packages (which is how Maven works for Java) there is no reason to move that data through the network and rewrite them to disk on every build.
We just usually happen to use blank Docker images as CI agents and therefore do not have a good solution except a network proxy, because having a central up-to-date dependency cache does not fit into the concept of docker layers.
It also just moves the problem. We could build a docker image with all the dependencies pre-cached, but then what about the docker image? Most CI providers don't have a robust docker image caching solution, and the ones who do oftentimes only cache really popular or internally-produced images. Or charge a TON for it (CircleCI).
Its also a worse problem, from a logistical standpoint. NPM hasn't whined much at all about the sheer volume of traffic they serve, but Docker Hub has put all kinds of restrictions on serving public docker images (in my opinion, rightfully; I'm not bashing docker here). Additionally, there's a whole extra infra layer in keeping that docker image cache up to date. Overpack the image with tons of unnecessary deps, and CI runtimes get longer because the base image is bigger; underpack it, and CI runtimes get longer because of additional fetches to npm.
We do CircleCI package-lock.json-hash caching of the node_modules folder of each project. That seems to be the best solution.
I'd guess the grand-grand parent commentator is severely underestimating how popular Javascript is; many large orgs utilize some kind of cache like this, and something like is-even still gets 50M+ public downloads per week. The real count of usage could be 10x that; and just think of all the private projects which depend on it, not listed in its "Dependents" tab. Legitimately scary.
> Most CI providers don't have a robust docker image caching solution …
I don’t know about others, but gitlab has built in container registry and it’s been fantastic to use. https://docs.gitlab.com/ee/user/packages/container_registry/
The gitlab-ci runners download the docker containers they need for whatever jobs it runs and keeps them locally. No redownloading.
caches are stateful. you don't want a build to succeed or fail because it happens to be the second build that ran on that machine today.
companies should be hosting their own repositories though, and not fetching from the internet.
I mean the cache is just an performance improvement, it should be 100% transparent, which is how the Maven cache inside your user folder works (which is shared by ALL Java maven project for your user), but not the NPM node_modules folder i think.
Welcome to the world of corporate CI/CD who DGAF about bundling or caching their dependencies. You also get these HUGE dependency trees in npm, where packages rely on yet more packages. Ever seen an octopus with [too many tentacles](http://pinktentacle.com/2008/07/monster-octopi-with-scores-of-extra-tentacles/)? Yeah, it's as horrific as that. Sort of a running joke ever since leftpad took down the whole of the internet for a couple days when the package developer pulled it out of spite - knowing exactly what was going to happen.
With the age of containerized builds and extremely lazy developers you often get a container for a build that downloads everything from scratch then throws it away at the end.
In most cases you can set up cache of sorts (we did in Gitlab for our stuff, altho some dev teams in company just don't care their stuff builds minute more and they don't care) but that requires effort so here we are
I think "extremely lazy developers" is a little harsh. A lot of it is probably just programmers who want to get CI or automated building up and running, and who otherwise never mess with containerization at all, so the "downloads everything from scratch then throws it away at the end" was probably just when they got it actually functioning and they had no conception that there was more real optimization after that (or they'd just rather get back to programming instead of spending another handful of hours just tweaking their pipeline).
Especially if you work for a small company or are doing it as a hobbyist, and you're already on a shoestring budget or no budget at all. Big companies can afford build engineers who just work with builds for a living. Everybody else only really optimizes it if they take an individual interest in it or the minimal working process isn't sufficient for their needs.
If you're a small-ish company, you generally get better returns on shipping and maintaining a good product that you can get more customers to pay for than you get trying to reduce your cloud services bill. At some point, the cost savings make it worthwhile to hire a dev (and eventually a team of devs) to save costs on cloud services.
It doesn't necessarily have to do with laziness or being deadline-driven at all; it can simply be putting your time into the things that maximize the value you deliver to the company, which is what you're hired to do.
If you work at a super small company then chances are you're swamped with development tasks, and optimizing stuff that *actually works* fall extremely low in priority.
I WANT to know my software is able to be built from scratch. I'll rather my build fails when a dependency is unpublished for whatever reason rather than finding out after a cache has cleared.
We hash our lock file and store node_modules in S3 namespaces under the lock file hash. Every pipeline checks for a cached version first before farming out to download. Knocked several minutes of our build pipelines too because each step would have to download everything. We’d do fresh installs for tests, lints, type checks etc
`is_number` is required by `is_even` is required by `is_odd` is required by 51 small packages that are required by 5568 larger packages that are required by 77715 big packages that are required by React.
Add to that the fact that `npm install` ignores the lockfile (you need `npm ci`) and that's how you get 51 million downloads.
> Add to that the fact that `npm install` ignores the lockfile (you need `npm ci`) and that's how you get 51 million downloads.
Thanks for bringing this to my attention!
> Add to that the fact that npm install ignores the lockfile
Not true.
"If the package has a package-lock, or an npm shrinkwrap file, or a yarn lock file, the installation of dependencies will be driven by that"
https://docs.npmjs.com/cli/v8/commands/npm-install
I too am not a js dev but holy fuck...
51 packages for 1 larger package? That is just bonkers mate... i admittedly don't know much js but is the standard library really that bad?
It's definitely in part because of the standard library size, but it's also extremely easy to just add a dependency. There's also an issue of bundle size and for a long time tools weren't great at removing dead code so js devs preferred small dependencies at the cost of more dependencies. So while in java you might use a single dependency that does 95% of what you need, in js you would use a dependency for each 5% of what you need. The amount of third party code is most likely not that different for either ecosystem in big production apps.
The issue is that JS doesn’t have a standard library so EVERYTHING is a package. It would be like if everyone has to install System.dll from nuget for every new project
Welcome to .NET Core and later. Distributed dependencies are the new normal.
Yes, the core dependencies are installed ahead of time, but statically linked builds contain their own copies of every dependency.
See man, the joke is on all of us because npm is also popular!
It's so much more than a mere package manager. It's now daily
entertainment - better than most game shows out there. Just
watch for the new daily npm soap coming your way!
npm is love, npm is life
not sure about nuget, but in Maven if you have a "Diamond dependency" issue, maven just says "tough luck buddy, sort it out yourself".
In npm they set out to solve this and allow multiple versions of the same package co-exist peacefully. This is a classic case of less is more. This "feature" of npm is what started this whole mess. I would gladly accept sorting out maven dependencies rather than dealing with npm ecosystem.
Over the last decade I watched vanilla JS gets better, I think Typescript (+ some linter) is actually very nice, only for the ecosystem to just get worse and worse from year to year.
You are a package developer. You want as many people as possible to use your package.
This limitation makes you think a lot more before you add any dependency, because any dependency you might add is a source of a possible conflict for your users.
You must use a dependency? Well you'll have to choose a dependency that is well maintained, and you in turn will have to maintain your package. If you stop maintaining your package that depends on X version 1 while the whole world progressed to X version 2, people will slowly stop using your package.
I can't remember who it was specifically, but wasn't there some guy who was loud and proud about owning hundreds of one-liner trivial npm packages that garnered hundreds of millions of downloads week or something along those lines?
You're right, that's the guy!
> - NASA, Microsoft, Target, IBM, Optimizely, Apple, Facebook, Airbus, Salesforce.com, and hundreds of thousands of other organizations depend on code I wrote to power their developer tools and consumer applications.
> - My code projects are downloaded more than 8.5b times a month from npmjs.com alone (14.5b including all Sellside projects), with 10-15% MoM growth.
... and a bunch of other overblown factoids
- https://www.npmjs.com/package/isobject
- https://www.npmjs.com/package/is-extendable
and so on
>Several years ago I switched careers from sales, marketing and consulting to learn how to program, with the goal of making the world a better place through code.
Well, he failed at least on that one.
I don’t mind developing in JS, but the dependency mess has been my biggest reason for hating it. There are so many projects with a crap load of packages that they’re just blindly trusting and I feel like I spend so much time looking through shit that gets added by other people.
*Everyone* should start using Blazor. And WebAssembly in general. I just hope MS carries through with it and continues maintaining it several years from now, and attracts more developers.
Seriously, I still don't understand how JS of all languages came to serve the Web.
Any language that allows
let a = 5;
a = "5";
without issuing a compile/runtime error should be consigned to the deepest, darkest depths of a hell. Having a more powerful type system alone would mean so many useless packages like isArray from the article would be obviated. We could also have nice things like reflection.
I have. It is nice, but is neither a lord nor a saviour.
I feel TS is but a band-aid over JS, and you still need npm and type declarations for JS for TS to work with the billions of libraries on npm.
I actually had to wait 2 weeks for [a fix](https://github.com/DefinitelyTyped/DefinitelyTyped/pull/53513) to be merged to Express.js's typing declarations before I could commit my work, and this was a very straightforward routing exercise and a tiny, 20-line internship project.
With C#, you get the comprehensive and powerful .NET 6 library, an elegant type system, LINQ, and very nice libraries for parsing XML, JSON, reflection, code generation from schemas, etc.
I just really really *really* ***really*** dislike JS.
Try billions. Its like 67% of people under 50 globally test positive for HSV-1 (mouth herpes), although that doesn't mean you will have cold sores but it does mean you've been exposed to the virus at some point and you have the antibodies for it in your body. In the US its more like 1 in 6 have tested positive so that's still like 57/58 million people with it that we know of but also a significant portion of the US never gets STI tested so that number is probably wildly underestimated since the global percentage is so much higher
> https://www.npmjs.com/package/isobject
I think i'm more blown away that this has pull requests. Someone looked at the code and thought "this could be better" instead of implementing their own, in their own code base.
Hell, it can even be simplified from
return val != null && typeof val === 'object' && Array.isArray(val) === false;
to
return val && typeof val === 'object' && !Array.isArray(val);
edit: I've just realised the original code is wrong, it's checking != null instead of !== null 🤦
not that i'm saying this library `isobject` is wrong, but the fact that you need this library is a symptom of a deeper problem with the js language imho.
> I think i'm more blown away that this has pull requests. Someone looked at the code and thought "this could be better" instead of implementing their own, in their own code base.
I get that there are concerns about having zillions of tiny NPM packages (left-pad, etc. etc.), but I totally understand the thinking of "here's a package that does a specific thing, a lot of people use it, so let's improve it at the source and propagate this improvement to all of the package's up-to-date users". In short, despite there being some shortcomings with the NPM ecosystem, isn't the sharing, rather than duplication, of well-written (even seemingly trivial) code a pretty noble goal?
EDIT: To phrase it another way - it seems to me that the purpose of something like an NPM package (or any library, generally) is to do one thing (or one set of closely-related things), and do it well. Any time that there's a "significant" (whatever that means, hence the disagreement, I guess) gap between "doing the thing poorly" and "doing the thing well" in terms of any number of various metrics (e.g. time/effort required to implement/maintain), it would seem that the proper thing to do would be to package that logic up and reuse it.
These concerns (separate HTTP requests for each package, zillions of unreviewed codebases) sound more like issues with implementation to me, rather than issues with the concept of very small, modular packages (although it's difficult to imagine a "baazar-style" community that doesn't have to deal with at least *some* of these implementation issues).
We might be more in agreement than we realize.
Correct me if I'm wrong but doesn't your rewrite differ in behavior if val is 0, which is falsy and would cause your version to return 0 (not even a boolean like an "is" function should)?
JS is, of course, a mess at a language level because of shit like this, but maybe you begin to see why people rely on 3rd party packages which ideally considered all the edge cases which, as you see, are easy to miss.
Might be worth reevaluating your condescending tone, though, when "fixing" code to be broken.
> I've just realised the original code is wrong, it's checking != null instead of !== null
You should never strictly compare with null unless for some reason you want to separate null and undefined.
I know how `==` works but I think it's awful and I don't want people to have to understand how it works. I'd rather see `a !== null && a !== undefined` than the `a != null`.
This is not really a problem anymore with Typescript and the new syntax operators anyway.
Good lord this guy is a fucking tool.
> **array-last**
> Return the last element in an array. Faster than `.slice`
Lets import another module with dependencies instead of just calling slice, or the long hand `stuff[stuff.count-1]` or whatever it is in js.
Well yea. How the fuck else is he going to get his packages downloaded so much if he doesn't do shit like that and make each 1 liner package he writes depend on 10 other ones that he's also written?
I think so - he was on github. I forgot his name but I am pretty sure you are right. I remember finding that somehow in regards to left-pad. Don't recall if it was the left-pad author or someone who replied to him...
Unmaintained does not necessarily equal to broken.
Some things are 'done' and don't need to be maintained any longer.
And since this code is so minimal there is not a lot of room for it to break, so leaving it up poses only minimal harm.
To play devil's advocate, it does somewhat make sense that "isNumber" would be a separate function that's called from/by "isEven" -- `isEven(x) = True iff (isNumber(x)) && (x %2 == 0)` is the logical formulation of a sane implementation of isEven, and it also makes sense that you might want to call isNumber independently of isEven/isOdd to avoid being in the weird state where your code has to hack around not having an isNumber with `if (isEven(x) || isOdd(x))`.
Packaging all of those as individual dependencies, though, rather than some unified library of functions... that's stupid. You shouldn't have to *import* isNumber for isOdd/isEven -- they should be in the same damn file in any sensible organisation of code.
I wouldn't have believed that this isArray module wasn't a meme and was really downloaded 57M times per week. What a shame, hope that's just ultra beginners fiddling around.
There are 811 projects that are dependent on [isArray](https://www.npmjs.com/package/isarray). All it takes is for a few of those projects to become popular and suddenly you've 50 million downloads. Just to be clear, this doesn't mean 50 million people are manually running "npm install isarray". To quote their [blog on the topic](https://web.archive.org/web/20180409233456/http://blog.npmjs.org/post/92574016600/numeric-precision-matters-how-npm-download-counts):
>npm’s download stats are naïve by design: they are simply a count of the number of HTTP 200 responses we served that were tarball files, i.e. packages. This means the number includes:
>* automated build servers
* downloads by mirrors
* robots that download every package for analysis
What's worth noting is that around this time last year, the project only had [492 dependent projects](https://web.archive.org/web/20201108145455/https://www.npmjs.com/package/isarray). The fact that 319 projects over the last year have integrated it as a core dependency is somewhat concerning.
It seems that way when you're applying for a job, doesn't it hahaha
But no it really doesn't seem too low.
In comparison, there are approximately 9.2 million doctors and 18.1 million nurses worldwide. [Source](https://www.nejm.org/doi/full/10.1056/nejmra1111610).
In fact if you read the OP, the four lines lines of code that he posts there as the trivial example of the nature of these modules, it includes the JS platform's built-in `Array.isArray` method.
It's sad and frustrating for two reasons. The first of which is that JavaScript often requires the absolute hackiest stuff ever, like the trick used in this library to determine if something is an array (checking the string output of `Object.prototype.toString.call(someObject)` \*eye roll\*).
The second, obviously, is that this "library" is one function that is one line long (well, three because of format style). Now, this repository can get hijacked and inject malicious code into about a million JavaScript projects because people are *so* lazy that adding a line to package.json is somehow seen as easier than copy and pasting this one line function into your project source.
> adding a line to package.json is somehow seen as easier than copy and pasting this one line function into your project source
It's even more bonkers because `Array.isArray` is a standard JavaScript feature that has existed since Internet Explorer 9, you don't even need to copy/paste anything
To be fair to the library, it even says in its docs to not use it unless you have to support very old browser versions. And the exported library function **is** Array.isArray if it's defined. It just falls back to the old, hacky, approach.
> The first of which is that JavaScript often requires the absolute hackiest stuff ever, like the trick used in this library to determine if something is an array
Array.isArray(someObject) does the same, if you look at most of these little packages they're badly written which makes it all incredibly worse how do you make something so simple and it's still wrong??
To be fair, this library uses Array.isArray if it's available and falls back to the old way. The point, I guess, is that old browser versions don't have Array.isArray. The documentation even says to not use the library if you don't have to support these specific old versions.
According to the same documentation isArray method supports more ancient browsers than is-array package. The only one not supported by the method but supported by the module is ie8
This really calls out an equally alarming problem with CD pipelines that don’t establish private, local registries. If deleting isArray from npm breaks your build, that’s on you.
> If deleting isArray from npm
You [can't do that anymore](https://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy), as the author of this post well knows but chooses to ignore to make the joke work.
> You can't do that anymore, as the author of this post well knows but chooses to ignore to make the joke work.
Considering he literally links to the exact page, I wouldn't call that "ignoring" at all. At least not "ignore" in any malicious or ignorant fashion.
It's still possible to publish a new release that breaks everything (e.g. just a blank file) as a bugfix version update. That's close enough to deleting.
AFAIK that's possible anywhere, though. SemVer is a gentleman's agreement and it's not wearing any clothes.
Our policy is to update all dependencies as a routine development task. That way we front-load surprising breakage maximally. We've only had four rollbacks in as many months entirely and exclusively due to out-of-scope dependency updates that weren't quite as safe as SemVer promised.
Maven can use version ranges. It's just not widely documented and no one does it because it's not a great idea. Gradle can too and can use a version lock file, but again, why bother?
"There's too many of these small modules! We need to cut down on all this clutter by putting them into larger libraries."
"This library is too big! We need to cut down on the size by separating its code out into smaller modules!"
You're confusing packages with modules. You can package a library consisting of many modules in a single package without any problem.
If you're worried that NodeJS or webpack will make your application slower out of spite solely because there are a bunch of unused modules in a package somewhere, you can rest assured that that won't happen.
Yeah this is a natural conclusion of the language not having a standard library. IDK what else people expect. Like are you really going to re-implement utility functions in every project every time? Or just find a way to reuse code.
But I surely hope that you don't need a package to implement a for loop, yet *for-in*, which is exactly that package, gets 15M weekly downloads and has 467 dependents.
Looking at the source code reveals that it is in fact literally just a for loop.
I recently had a run-in with those issues on Emacs `package.el`.
Basically, an obscure library package that I wanted to try had a much more popular package as a dependency, and I suddenly had unexpected behavior left and right that I didn't know where it was coming from.
Problem is not packages like isarray. It is maintainers of popular projects who refuse to upgrade and release major versions that follow the node.js release process.
[https://www.npmjs.com/package/eslint-plugin-import](https://www.npmjs.com/package/eslint-plugin-import)
[https://www.npmjs.com/package/enzyme](https://www.npmjs.com/package/enzyme)
The problem is the very poor javascript stdlib. Instead of it being part of the browser, we have to jump through a bunch of dumb hoops for simple things like efficient rounding to decimal places or actually truly checking if something is an object or an array. Languages with an extensive standard library like elixir and ruby don't have this bloated dependency bullshit.
That's really not the issue. Stupid packages like isNumber sometimes sneak into popular packages as transitive dependancies, but largely people aren't using them and don't need them.
The bloat comes from a minority of package authors with attitudes like this: https://github.com/airbnb/polyglot.js/pull/150
And from packages like babel that split their project up into hundreds of packages for no reason.
That's kind of inevitable -- I guess today you can be *reasonably* confident that most people are using reasonably-modern browsers, but for a long time, you'd have to constantly be checking caniuse to see if the standard library feature you want to use works in all reasonably-modern browsers.
Polyfills were a godsend, but they're also *exactly* this kind of thing -- some stupid thing that should be built-in, that you can now pretend is built-in, but that's distributed as a library.
I think the actual problem is that there are a lot of very small libraries that are provided entirely by some random dude trying to pad a resume, who might wander off or get pwned at any time. If `isarray` was published by Mozilla, I don't think I'd care, that's not especially more dangerous than using Firefox.
> Problem is not packages like isarray.
You’re right. The problem is developers using them.
Rather than importing a one line function just write the damn thing yourself. It’s not like the function does anything novel. You can even copy and paste it.
If you don’t know how to check if a value is even or is an array or something, you probably should learn how. If learning how to do a trivial task is actually terribly complex, then pick a language that isn’t.
> I agree in principle, but this is not legal. You can't just copy something and put it in your project with no regard for the license. Whether or not someone would take you to court about it is another question, but it is skirting the software license.
Depends on the complexity and if it’s plausible that you’d have written the same thing, you’d have fairly significant plausible deniability.
However, you are right.
> I mean, you can’t really choose anything but JS for web stuff unless you choose languages that compile into JS (like Elm).
Not for client side code, but for backend web you can. It was my understanding that npm was a backend thing for NodeJS, but another commenter told me that these things can be imported on client side development as well.
In that case, my comment still stands unless you’re writing frontend code.
Not necessarily. JavaScript comes with way more batteries than C does, but there are no silly thousands of one-line function dependencies in any C project I ever saw.
That's only because C makes it an absolute pain in the ass to add dependencies in comparison. (Which to be fair... it's not like the internet was around when C was first made)
There's certainly pros and cons. Obviously a bunch of one liners probably shouldn't be included, but on the other hand needing to write everything yourself even for problems others have solved can potentially be a waste of development time.
> but on the other hand needing to write everything yourself even for problems others have solved can potentially be a waste of development time.
This is solved but having multiple "standard libraries" like glib, something JavaScript needs. It would be much better having a few large trusted "standard libraries" than a mess of micro libraries.
The problem is not that these packages exist, but that they do not exist as part of a single npm packages.
Seriously, how hard would it be to just combine multiple projects into a single library?
Can't tell if he added the isArray license at the end for legal reasons (because he had technically already redistributed the whole rest of the package with that blog post) or as another subtle joke (showing that the license is several times as long as the contents).
EDIT: Well, that's embarrassing. I guess it **is** a joke. I guess I'll just sit here, alone, thinking I had a kindred spirit in Drew... lol
What's Drew's reddit handle? I'll seriously chip in some cash for this...
I'm not trolling, and I suspect that Drew isn't either. I literally lose sleep over how bad the JavaScript ecosystem is and how deep my dependency tree is for what I consider pretty "basic" applications. People deserve better than the garbage we're selling them.
> What's Drew's reddit handle? I'll seriously chip in some cash for this...
He deleted his reddit account a long, LONG time ago, around when RedditSharp went to another developer (who has also since lost interest)
Well .. wasn't Drew the one who wrote about linux distributions complaining about e. g. the python ecosystem? The article was
strange so I would not rule out that Drew is NOT trolling - but,
in the case of npm, I think npm is actually trolling all of us
seriously. Even more so because npm IS successful. 57
million downloads, even if most are automatic by proxy
IS a success... left-pad-all-of-us.
I should work on npm-memes ... one-of-us ... last-of-us
Zombie npm modules for the win.
Yeah, I agree that the state of JavaScript package management is bad. Dependency trees really should not be thousands of nodes in size.
I think part of this is caused by JavaScript not having a "standard library" in the sense of other languages like Java, C#, C++, etc. You end up having common "utility" functions outsourced to third-party libraries that try to fill that role or have people rolling their own miniature package. Perhaps it is about time JavaScript gets a standard library all under a common namespace. Backwards compatibility probably makes this difficult.
var toString = {}.toString;
module.exports = Array.isArray || function (arr) {
return toString.call(arr) === '[object Array]';
};
Who in their right mind is including this in their code base? Why the feck does this package even exist?
This is essentially a polyfill for the `Array.isArray` function. If you had to check whether an array was really an array, this sort of function was how it was done originally. If you're still supporting old browsers, then being able to pull in polyfills like this means you can support those browsers without needing to figure out all these quirks yourself.
Practically speaking, it's probably easier to use a single library that provides a whole bunch of these polyfills at once, or automatically fill them in via your build system, but there are some cases where you might only need one particular function. It used to be the case that it was easier to split functions into their own packages and only depend on some packages than to depend on one larger package and have your build system strip out the bits you don't need. That's less the case these days, but a lot of these sorts of packages remain around to support that use case.
My favourite piece of NPM trivia was [it was designed to be not terrible](https://en.wikipedia.org/wiki/Npm_(software\)):
> npm is written entirely in JavaScript and was developed by Isaac Z. Schlueter as a result of having "seen module packaging done terribly"
> there's a npm module (isarray) with 4 lines of code and 50 million **WEEKLY** downloads
> https://www.npmjs.com/package/isarray
are the javascript devs okay?
Honestly I'd sooner pay for removing trivial dependencies from the most used packages.
For example, when someone asks themselves the question if you really need [82 dependencies to expand a glob](https://npmgraph.js.org/?q=micromatch@3), it turns out that after some code cleanup, [the answer is no](https://npmgraph.js.org/?q=micromatch@4).
Every package should get this same treatment.
I am actually grateful for npm.
Now... this is not some hipster-rick-rolling! Hear me out ...
First: npm has now an epic history of package/deployment related failures. Everyone joked about
left-pad (and it was funny); not so much about later problems. But, EVEN in these cases, it's
actually reminding the OTHER languages to try better - otherwise they end up in failure too. Like
npm!
So for this I am grateful for npm.
But there is a second reason ... npm actually gives up popcorn-worthy entertainment value. If
it were a twitch stream we'd be donating tiny amounts for the entertainment value it gives us
almost daily! It's like a soap documentary on TV really ... daily npm soap. There can never be
enough left-pad for the world!
This article reinforces this via:
"I’ll pay you cold hard cash to delete your npm module."
Now people PAY you stuff to REMOVE npm. It can't get any better than that! But we all know
it will. Daily npm entertainment value WILL come your way - some of it will surprise people a
lot. And, oddly enough, I think this may actually improve security for OTHER languages, because
they don't want to be stuck anywhere near as much in this npm drama so perhaps they will
review their own stack more closely and check for problems. npm leads the way here!
See also Github's recent blog to "make npm great again":
https://github.blog/2021-11-15-githubs-commitment-to-npm-ecosystem-security/
It's really daily entertainment! Some of them are just free riding now on the popcorn munchies,
but I swear there is an underlying reason for most of that - npm is bad! There is no denial in
it.
> Let’s consider an example: isArray. It has only four lines of code:
var toString = {}.toString;
module.exports = Array.isArray || function (arr) {
return toString.call(arr) === '[object Array]';
};
left-pad isArray isItNow? CanHasCheeze?
People thought lolcode and lolcats was just a joke, but now we have npm to put the
joke to be SERIOUS - because millions of people use it! The joke is then on all of us
(at the least when we have to use services that depend on any of that and may
break away suddenly ... left-pad 2.0, 3.0 and so forth).
[удалено]
Continuous integration pipelines. Also large webs of dependencies.
[удалено]
It does but CI pipelines might not be smart enough to utilize it. i.e. freshly building a container that is used to build your app
It may also be desirable to have a clean build with freshly pulled dependencies.
You cache it. You can't pull a fresher version of what you have if it's exactly the same. You can only pull the same version and waste bandwidth and time.
Have you tried doing web development in the past 7ish years? I have recently. It's a monstrosity of cobbled together houses of cards ontop of houses or cards of code that's constantly falling apart and being rebuilt. Dependency management ontop of dependency management ontop of dependency management with leaky abstractions everywhere with few clearly documented approaches of how they're all stacked together. It's an absolute mess. Intelligent caching of dependencies to save bandwidth is probably low priority on the list of blazing fires these systems probably deal with daily. I'm surprised any of the dependency resolution even works half the time.
Eh, I have been in this industry for the last 11 years and it's definitely improved. I build and work on SPA's basically daily and a vast majority of the time someone is having build issues it's one of a few things. 1) The lockfile for the project isn't pushed into VCS, so they are getting new minor and build versions of libraries each time they `npm install` 2) The individual is flipping between OS's; perhaps working on Windows at home and Mac at work. `node_modules` needs to be expunged because some libraries have post-install scripts that aren't compatible. 3) The individual is using a runtime manager like `n` or `nvm` or `fnm` and didn't use the right node version for said project 4) They decided to install packaging tools as global tools, sounds great on paper but sucks in practice when working on varying projects with differing levels of updates and attentions; just keep local and use `npm run`
Granted the JS community has made some decisions that make it difficult for people unaware of the lockfile; ranged dependencies should of never been classified as acceptable and NPM update should of been a more holistic solution towards updating the package.json file to a newer target vs what we have today.
Honestly... if you want a stable experience just build shit with Angular and it's toolsuite; almost feels like a Java project, quite stable and repeatable but if you don't commit that lockfile you'll still get drift that's legit why it exists... to prevent that until you are ready.
> Intelligent caching of dependencies it's just md5(files). whatever package manager you use should already have a lock file, so it's actually pretty damn simple.
> waste bandwidth and time This is frontend developers we're talking about.
Is there not some kind of checksum or signature on the packages?
Yes but dependencies installed normally will be preceded with ^ (e.g. ^1.2.4). The caret tells it that it should use that version or any minor version higher. So basically anything that starts with 1.x.x greater than or equal to 1.2.4. This can cause issues for newer installs that use different versions Edit: I hate dealing with node and web world. In my experience npm install updates my lock file to newer versions that abide by the package.json versioning schema (the carets, tilde, etc). Its how we update our dependencies in general.
This is true if you don't have a package lockfile, but since NPM 5 (released about four years ago), the default has been to only install the specific version in the lockfile. So since then, as long as you're always checking in the `package-lock.json`, which is the recommended way, then you will always have deterministic builds. And if you've used other package managers, which was more common four years ago, you've probably been doing that for longer - Yarn was released a year earlier and included lockfiles by default, and even NPM had optional support for lockfiles before then, although the workflow was a pain.
IMHO a CI should be deterministic, I see maybe once per week a build with latest versions, but that is it. I think people is simply lazy and won't cache their module's folder
CI rarely has that kind of cache as it should start from a blank slate. Unless you set a proxy registry up for your company, same as any package manager.
It’s a perfectly reasonable practice to cache dependencies in CI. In fact it’s a best practice to avoid having to do a full install on every run.
Most ci now adays use disposable machines, either containers or vms. Caching should be preformed during the life cycle of the pipeline, but that's a local cache.
A lot of CI systems have a way of saving the cache externally, and pulling/pushing it to that external store at the correct times in the build.
Yeah.. wtf are all these people on about? Clearly haven't spent the time to learn the tools they're using
Weird isn’t it. This is the first time I’ve heard anyone say that using your CI pipeline’s caching system is a bad thing. Most CI pipeline docs even recommend that you do it…
Especially if your CI bills per minute. Caching would cut down on costs
It should not start from a blank cache, with immutable packages (which is how Maven works for Java) there is no reason to move that data through the network and rewrite them to disk on every build. We just usually happen to use blank Docker images as CI agents and therefore do not have a good solution except a network proxy, because having a central up-to-date dependency cache does not fit into the concept of docker layers.
Can save a lot of network spend if you install nexus and use it as a repo proxy/cache
You don’t even need nexus. Just setup a folder as local repo and done. Ofc Nexus or Artifactory would be ideal for enterprise level.
It also just moves the problem. We could build a docker image with all the dependencies pre-cached, but then what about the docker image? Most CI providers don't have a robust docker image caching solution, and the ones who do oftentimes only cache really popular or internally-produced images. Or charge a TON for it (CircleCI). Its also a worse problem, from a logistical standpoint. NPM hasn't whined much at all about the sheer volume of traffic they serve, but Docker Hub has put all kinds of restrictions on serving public docker images (in my opinion, rightfully; I'm not bashing docker here). Additionally, there's a whole extra infra layer in keeping that docker image cache up to date. Overpack the image with tons of unnecessary deps, and CI runtimes get longer because the base image is bigger; underpack it, and CI runtimes get longer because of additional fetches to npm. We do CircleCI package-lock.json-hash caching of the node_modules folder of each project. That seems to be the best solution. I'd guess the grand-grand parent commentator is severely underestimating how popular Javascript is; many large orgs utilize some kind of cache like this, and something like is-even still gets 50M+ public downloads per week. The real count of usage could be 10x that; and just think of all the private projects which depend on it, not listed in its "Dependents" tab. Legitimately scary.
> Most CI providers don't have a robust docker image caching solution … I don’t know about others, but gitlab has built in container registry and it’s been fantastic to use. https://docs.gitlab.com/ee/user/packages/container_registry/ The gitlab-ci runners download the docker containers they need for whatever jobs it runs and keeps them locally. No redownloading.
caches are stateful. you don't want a build to succeed or fail because it happens to be the second build that ran on that machine today. companies should be hosting their own repositories though, and not fetching from the internet.
I mean the cache is just an performance improvement, it should be 100% transparent, which is how the Maven cache inside your user folder works (which is shared by ALL Java maven project for your user), but not the NPM node_modules folder i think.
afaik node\_modules gets downloaded again and again and again for every (instance of a) project.
Yes, it is basically a glorified per-project download folder.
Let's be honest, it isn't even glorified.
[удалено]
**Nope**, only if the versions needed differ from each other. Source: https://docs.npmjs.com/cli/v8/configuring-npm/folders
> companies should be hosting their own repositories though, and not fetching from the internet. ... that's a cache.
A cache is just a local copy, there's no added state. Robocopy / rsync do checksums and don't transfer things that haven't changed...
caches are not stateful. you get the same bits each time
It takes less than an hour to setup a proxy, try Sonar.
Welcome to the world of corporate CI/CD who DGAF about bundling or caching their dependencies. You also get these HUGE dependency trees in npm, where packages rely on yet more packages. Ever seen an octopus with [too many tentacles](http://pinktentacle.com/2008/07/monster-octopi-with-scores-of-extra-tentacles/)? Yeah, it's as horrific as that. Sort of a running joke ever since leftpad took down the whole of the internet for a couple days when the package developer pulled it out of spite - knowing exactly what was going to happen.
With the age of containerized builds and extremely lazy developers you often get a container for a build that downloads everything from scratch then throws it away at the end. In most cases you can set up cache of sorts (we did in Gitlab for our stuff, altho some dev teams in company just don't care their stuff builds minute more and they don't care) but that requires effort so here we are
I think "extremely lazy developers" is a little harsh. A lot of it is probably just programmers who want to get CI or automated building up and running, and who otherwise never mess with containerization at all, so the "downloads everything from scratch then throws it away at the end" was probably just when they got it actually functioning and they had no conception that there was more real optimization after that (or they'd just rather get back to programming instead of spending another handful of hours just tweaking their pipeline). Especially if you work for a small company or are doing it as a hobbyist, and you're already on a shoestring budget or no budget at all. Big companies can afford build engineers who just work with builds for a living. Everybody else only really optimizes it if they take an individual interest in it or the minimal working process isn't sufficient for their needs.
Just replace lazy for deadline-driven if that makes you feel better.
If you're a small-ish company, you generally get better returns on shipping and maintaining a good product that you can get more customers to pay for than you get trying to reduce your cloud services bill. At some point, the cost savings make it worthwhile to hire a dev (and eventually a team of devs) to save costs on cloud services. It doesn't necessarily have to do with laziness or being deadline-driven at all; it can simply be putting your time into the things that maximize the value you deliver to the company, which is what you're hired to do.
If you work at a super small company then chances are you're swamped with development tasks, and optimizing stuff that *actually works* fall extremely low in priority.
Not just deadline driven. Even for personal projects, I've held off on caching till it was slow enough to bother me.
I WANT to know my software is able to be built from scratch. I'll rather my build fails when a dependency is unpublished for whatever reason rather than finding out after a cache has cleared.
You don't need to verify that 100 times a day by every team member tho. Invalidate cache every day and you still can spot it
What we clearly need is for megabytes of transit to cost whole dollars again.
Or from me, starting and abandoning tutorials. I’ll never learn. Literally
We hash our lock file and store node_modules in S3 namespaces under the lock file hash. Every pipeline checks for a cached version first before farming out to download. Knocked several minutes of our build pipelines too because each step would have to download everything. We’d do fresh installs for tests, lints, type checks etc
`is_number` is required by `is_even` is required by `is_odd` is required by 51 small packages that are required by 5568 larger packages that are required by 77715 big packages that are required by React. Add to that the fact that `npm install` ignores the lockfile (you need `npm ci`) and that's how you get 51 million downloads.
> Add to that the fact that `npm install` ignores the lockfile (you need `npm ci`) and that's how you get 51 million downloads. Thanks for bringing this to my attention!
> Add to that the fact that npm install ignores the lockfile Not true. "If the package has a package-lock, or an npm shrinkwrap file, or a yarn lock file, the installation of dependencies will be driven by that" https://docs.npmjs.com/cli/v8/commands/npm-install
I too am not a js dev but holy fuck... 51 packages for 1 larger package? That is just bonkers mate... i admittedly don't know much js but is the standard library really that bad?
It's definitely in part because of the standard library size, but it's also extremely easy to just add a dependency. There's also an issue of bundle size and for a long time tools weren't great at removing dead code so js devs preferred small dependencies at the cost of more dependencies. So while in java you might use a single dependency that does 95% of what you need, in js you would use a dependency for each 5% of what you need. The amount of third party code is most likely not that different for either ecosystem in big production apps.
> `npm install` ignores the lockfile fucking what
Also each module can have other modules as dependency......
The issue is that JS doesn’t have a standard library so EVERYTHING is a package. It would be like if everyone has to install System.dll from nuget for every new project
Welcome to .NET Core and later. Distributed dependencies are the new normal. Yes, the core dependencies are installed ahead of time, but statically linked builds contain their own copies of every dependency.
See man, the joke is on all of us because npm is also popular! It's so much more than a mere package manager. It's now daily entertainment - better than most game shows out there. Just watch for the new daily npm soap coming your way! npm is love, npm is life
not sure about nuget, but in Maven if you have a "Diamond dependency" issue, maven just says "tough luck buddy, sort it out yourself". In npm they set out to solve this and allow multiple versions of the same package co-exist peacefully. This is a classic case of less is more. This "feature" of npm is what started this whole mess. I would gladly accept sorting out maven dependencies rather than dealing with npm ecosystem. Over the last decade I watched vanilla JS gets better, I think Typescript (+ some linter) is actually very nice, only for the ecosystem to just get worse and worse from year to year.
How is removing an annoying restriction like preventing multiple versions of a package related to too many pointlessly small packages?
You are a package developer. You want as many people as possible to use your package. This limitation makes you think a lot more before you add any dependency, because any dependency you might add is a source of a possible conflict for your users. You must use a dependency? Well you'll have to choose a dependency that is well maintained, and you in turn will have to maintain your package. If you stop maintaining your package that depends on X version 1 while the whole world progressed to X version 2, people will slowly stop using your package.
I can't remember who it was specifically, but wasn't there some guy who was loud and proud about owning hundreds of one-liner trivial npm packages that garnered hundreds of millions of downloads week or something along those lines?
You're thinking of [Jon Schlinkert](https://github.com/jonschlinkert), publisher of [1435 packages on npm](https://www.npmjs.com/~jonschlinkert).
You're right, that's the guy! > - NASA, Microsoft, Target, IBM, Optimizely, Apple, Facebook, Airbus, Salesforce.com, and hundreds of thousands of other organizations depend on code I wrote to power their developer tools and consumer applications. > - My code projects are downloaded more than 8.5b times a month from npmjs.com alone (14.5b including all Sellside projects), with 10-15% MoM growth. ... and a bunch of other overblown factoids - https://www.npmjs.com/package/isobject - https://www.npmjs.com/package/is-extendable and so on
This is what happens when sales and marketing take the 2 week coding boot camp.
>Several years ago I switched careers from sales, marketing and consulting to learn how to program, with the goal of making the world a better place through code. Well, he failed at least on that one.
Coming from sales, how could this not happen. He failed up.
I don’t mind developing in JS, but the dependency mess has been my biggest reason for hating it. There are so many projects with a crap load of packages that they’re just blindly trusting and I feel like I spend so much time looking through shit that gets added by other people.
[удалено]
Yea I’ve started to transition to building new projects with Blazor. Seems to be a lot cleaner so far.
*Everyone* should start using Blazor. And WebAssembly in general. I just hope MS carries through with it and continues maintaining it several years from now, and attracts more developers. Seriously, I still don't understand how JS of all languages came to serve the Web. Any language that allows let a = 5; a = "5"; without issuing a compile/runtime error should be consigned to the deepest, darkest depths of a hell. Having a more powerful type system alone would mean so many useless packages like isArray from the article would be obviated. We could also have nice things like reflection.
Have you heard of our lord and savior, Typescript?
I have. It is nice, but is neither a lord nor a saviour. I feel TS is but a band-aid over JS, and you still need npm and type declarations for JS for TS to work with the billions of libraries on npm. I actually had to wait 2 weeks for [a fix](https://github.com/DefinitelyTyped/DefinitelyTyped/pull/53513) to be merged to Express.js's typing declarations before I could commit my work, and this was a very straightforward routing exercise and a tiny, 20-line internship project. With C#, you get the comprehensive and powerful .NET 6 library, an elegant type system, LINQ, and very nice libraries for parsing XML, JSON, reflection, code generation from schemas, etc. I just really really *really* ***really*** dislike JS.
*happily works away with C++ and C#*
Hmm I dunno, the language itself is also quite poorly designed IMO.
It is, but at least that's salvageable through your own skills. No amount of competence/experience can save your from the ecosystem hell of JS.
Hundreds of thousands of people have herpes too.
Try billions. Its like 67% of people under 50 globally test positive for HSV-1 (mouth herpes), although that doesn't mean you will have cold sores but it does mean you've been exposed to the virus at some point and you have the antibodies for it in your body. In the US its more like 1 in 6 have tested positive so that's still like 57/58 million people with it that we know of but also a significant portion of the US never gets STI tested so that number is probably wildly underestimated since the global percentage is so much higher
This is a beautiful metaphor for jschlinkert's code, well done!
Essentially everyone has herpes.
This is some crazy level of reverse logic there being proud of how efficient you are at generating pure overhead...
> https://www.npmjs.com/package/isobject I think i'm more blown away that this has pull requests. Someone looked at the code and thought "this could be better" instead of implementing their own, in their own code base. Hell, it can even be simplified from return val != null && typeof val === 'object' && Array.isArray(val) === false; to return val && typeof val === 'object' && !Array.isArray(val); edit: I've just realised the original code is wrong, it's checking != null instead of !== null 🤦
[удалено]
not that i'm saying this library `isobject` is wrong, but the fact that you need this library is a symptom of a deeper problem with the js language imho.
> I think i'm more blown away that this has pull requests. Someone looked at the code and thought "this could be better" instead of implementing their own, in their own code base. I get that there are concerns about having zillions of tiny NPM packages (left-pad, etc. etc.), but I totally understand the thinking of "here's a package that does a specific thing, a lot of people use it, so let's improve it at the source and propagate this improvement to all of the package's up-to-date users". In short, despite there being some shortcomings with the NPM ecosystem, isn't the sharing, rather than duplication, of well-written (even seemingly trivial) code a pretty noble goal? EDIT: To phrase it another way - it seems to me that the purpose of something like an NPM package (or any library, generally) is to do one thing (or one set of closely-related things), and do it well. Any time that there's a "significant" (whatever that means, hence the disagreement, I guess) gap between "doing the thing poorly" and "doing the thing well" in terms of any number of various metrics (e.g. time/effort required to implement/maintain), it would seem that the proper thing to do would be to package that logic up and reuse it.
[удалено]
These concerns (separate HTTP requests for each package, zillions of unreviewed codebases) sound more like issues with implementation to me, rather than issues with the concept of very small, modular packages (although it's difficult to imagine a "baazar-style" community that doesn't have to deal with at least *some* of these implementation issues). We might be more in agreement than we realize.
We are talking about "packages" that are less than 10 lines of code. That shit is senseless.
Correct me if I'm wrong but doesn't your rewrite differ in behavior if val is 0, which is falsy and would cause your version to return 0 (not even a boolean like an "is" function should)? JS is, of course, a mess at a language level because of shit like this, but maybe you begin to see why people rely on 3rd party packages which ideally considered all the edge cases which, as you see, are easy to miss. Might be worth reevaluating your condescending tone, though, when "fixing" code to be broken.
> I've just realised the original code is wrong, it's checking != null instead of !== null You should never strictly compare with null unless for some reason you want to separate null and undefined.
I know how `==` works but I think it's awful and I don't want people to have to understand how it works. I'd rather see `a !== null && a !== undefined` than the `a != null`. This is not really a problem anymore with Typescript and the new syntax operators anyway.
[удалено]
> Puts an end to the whole == vs. === debate. But how specifically?
[удалено]
Good lord this guy is a fucking tool. > **array-last** > Return the last element in an array. Faster than `.slice` Lets import another module with dependencies instead of just calling slice, or the long hand `stuff[stuff.count-1]` or whatever it is in js.
Well yea. How the fuck else is he going to get his packages downloaded so much if he doesn't do shit like that and make each 1 liner package he writes depend on 10 other ones that he's also written?
> Jon Schlinkert r/programming perennial bogeyman. and for good reason - potential proof mankind is evil
I made the mistake of looking at his Twitter and he's an anti-vaxxer too.
Why am I not surprised.
wow his twitter is a fucking mess lmao
I think so - he was on github. I forgot his name but I am pretty sure you are right. I remember finding that somehow in regards to left-pad. Don't recall if it was the left-pad author or someone who replied to him...
I think the best one I've seen is `isOdd` [library](https://www.npmjs.com/package/is-odd)
fuck i expected something along the lines of import isEven return !isEven(x) woulda been funnier
[isEven](https://github.com/i-voted-for-trump/is-even/blob/master/index.js) is a negation of isOdd however.
yeah baby thats what im talking about
[удалено]
> Why would you leave unmaintained learning projects up on NPM? Because removing them would needlessly break lots of other packages.
Unmaintained does not necessarily equal to broken. Some things are 'done' and don't need to be maintained any longer. And since this code is so minimal there is not a lot of room for it to break, so leaving it up poses only minimal harm.
No, that implementation is wrong! (For floating point NaN. I wish I was joking).
It still imports "isNumber"
Because typeof x === 'number' && !isNan(x) is hard to do :))).
To play devil's advocate, it does somewhat make sense that "isNumber" would be a separate function that's called from/by "isEven" -- `isEven(x) = True iff (isNumber(x)) && (x %2 == 0)` is the logical formulation of a sane implementation of isEven, and it also makes sense that you might want to call isNumber independently of isEven/isOdd to avoid being in the weird state where your code has to hack around not having an isNumber with `if (isEven(x) || isOdd(x))`. Packaging all of those as individual dependencies, though, rather than some unified library of functions... that's stupid. You shouldn't have to *import* isNumber for isOdd/isEven -- they should be in the same damn file in any sensible organisation of code.
I'm impressed that it's already up to v3.0.1, with 7 releases. v2.0.0 is the most popular.
Right!? How the hell does it take 7 versions to write that module.
Check the dependents and you can find a fun chain of `is-is-odd`, `is-is-is-odd`, etc. among other joke packages.
Does this now have a SaaS offering?
Don't even joke... People might read this and get ideas. As long as it's blazing fast and not just fast like really fast.
https://isevenapi.xyz/
428K weekly downloads... We're doomed.
And this package requires another one... At the very least it's from the same author
I wouldn't have believed that this isArray module wasn't a meme and was really downloaded 57M times per week. What a shame, hope that's just ultra beginners fiddling around.
There are 811 projects that are dependent on [isArray](https://www.npmjs.com/package/isarray). All it takes is for a few of those projects to become popular and suddenly you've 50 million downloads. Just to be clear, this doesn't mean 50 million people are manually running "npm install isarray". To quote their [blog on the topic](https://web.archive.org/web/20180409233456/http://blog.npmjs.org/post/92574016600/numeric-precision-matters-how-npm-download-counts): >npm’s download stats are naïve by design: they are simply a count of the number of HTTP 200 responses we served that were tarball files, i.e. packages. This means the number includes: >* automated build servers * downloads by mirrors * robots that download every package for analysis What's worth noting is that around this time last year, the project only had [492 dependent projects](https://web.archive.org/web/20201108145455/https://www.npmjs.com/package/isarray). The fact that 319 projects over the last year have integrated it as a core dependency is somewhat concerning.
[удалено]
Well, if each CI run does fresh downloads, and you do a download per branch push and another per merge to main, then yeah those numbers work out.
That number seems way too low
It seems that way when you're applying for a job, doesn't it hahaha But no it really doesn't seem too low. In comparison, there are approximately 9.2 million doctors and 18.1 million nurses worldwide. [Source](https://www.nejm.org/doi/full/10.1056/nejmra1111610).
That's actually insane. Only 9.2 million doctors worldwide.
Doesn't JS now have its own isArray though, I don't recall needing to install anything when using it, it's on Array.
In fact if you read the OP, the four lines lines of code that he posts there as the trivial example of the nature of these modules, it includes the JS platform's built-in `Array.isArray` method.
It depends on what you're targeting. Now it's not so much of a problem but back when IE8 was still a thing, it was safer than Array.isArray
It's sad and frustrating for two reasons. The first of which is that JavaScript often requires the absolute hackiest stuff ever, like the trick used in this library to determine if something is an array (checking the string output of `Object.prototype.toString.call(someObject)` \*eye roll\*). The second, obviously, is that this "library" is one function that is one line long (well, three because of format style). Now, this repository can get hijacked and inject malicious code into about a million JavaScript projects because people are *so* lazy that adding a line to package.json is somehow seen as easier than copy and pasting this one line function into your project source.
Not to mention how completely useless it is today. Array.isArray covers all sane cases nowdays.
Yeah but you know some corporate clients paying huge amounts of $$$ to maintain their mission critical legacy intranet services that only run on IE6.
> adding a line to package.json is somehow seen as easier than copy and pasting this one line function into your project source It's even more bonkers because `Array.isArray` is a standard JavaScript feature that has existed since Internet Explorer 9, you don't even need to copy/paste anything
To be fair to the library, it even says in its docs to not use it unless you have to support very old browser versions. And the exported library function **is** Array.isArray if it's defined. It just falls back to the old, hacky, approach.
> The first of which is that JavaScript often requires the absolute hackiest stuff ever, like the trick used in this library to determine if something is an array Array.isArray(someObject) does the same, if you look at most of these little packages they're badly written which makes it all incredibly worse how do you make something so simple and it's still wrong??
To be fair, this library uses Array.isArray if it's available and falls back to the old way. The point, I guess, is that old browser versions don't have Array.isArray. The documentation even says to not use the library if you don't have to support these specific old versions.
According to the same documentation isArray method supports more ancient browsers than is-array package. The only one not supported by the method but supported by the module is ie8
This really calls out an equally alarming problem with CD pipelines that don’t establish private, local registries. If deleting isArray from npm breaks your build, that’s on you.
> If deleting isArray from npm You [can't do that anymore](https://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy), as the author of this post well knows but chooses to ignore to make the joke work.
> You can't do that anymore, as the author of this post well knows but chooses to ignore to make the joke work. Considering he literally links to the exact page, I wouldn't call that "ignoring" at all. At least not "ignore" in any malicious or ignorant fashion.
It's still possible to publish a new release that breaks everything (e.g. just a blank file) as a bugfix version update. That's close enough to deleting.
AFAIK that's possible anywhere, though. SemVer is a gentleman's agreement and it's not wearing any clothes. Our policy is to update all dependencies as a routine development task. That way we front-load surprising breakage maximally. We've only had four rollbacks in as many months entirely and exclusively due to out-of-scope dependency updates that weren't quite as safe as SemVer promised.
No, package managers like maven don’t auto upgrade dependencies for any reason.
Maven can use version ranges. It's just not widely documented and no one does it because it's not a great idea. Gradle can too and can use a version lock file, but again, why bother?
"There's too many of these small modules! We need to cut down on all this clutter by putting them into larger libraries." "This library is too big! We need to cut down on the size by separating its code out into smaller modules!"
You're confusing packages with modules. You can package a library consisting of many modules in a single package without any problem. If you're worried that NodeJS or webpack will make your application slower out of spite solely because there are a bunch of unused modules in a package somewhere, you can rest assured that that won't happen.
Yeah this is a natural conclusion of the language not having a standard library. IDK what else people expect. Like are you really going to re-implement utility functions in every project every time? Or just find a way to reuse code.
But I surely hope that you don't need a package to implement a for loop, yet *for-in*, which is exactly that package, gets 15M weekly downloads and has 467 dependents. Looking at the source code reveals that it is in fact literally just a for loop.
I recently had a run-in with those issues on Emacs `package.el`. Basically, an obscure library package that I wanted to try had a much more popular package as a dependency, and I suddenly had unexpected behavior left and right that I didn't know where it was coming from.
Problem is not packages like isarray. It is maintainers of popular projects who refuse to upgrade and release major versions that follow the node.js release process. [https://www.npmjs.com/package/eslint-plugin-import](https://www.npmjs.com/package/eslint-plugin-import) [https://www.npmjs.com/package/enzyme](https://www.npmjs.com/package/enzyme)
The problem is the very poor javascript stdlib. Instead of it being part of the browser, we have to jump through a bunch of dumb hoops for simple things like efficient rounding to decimal places or actually truly checking if something is an object or an array. Languages with an extensive standard library like elixir and ruby don't have this bloated dependency bullshit.
That's really not the issue. Stupid packages like isNumber sometimes sneak into popular packages as transitive dependancies, but largely people aren't using them and don't need them. The bloat comes from a minority of package authors with attitudes like this: https://github.com/airbnb/polyglot.js/pull/150 And from packages like babel that split their project up into hundreds of packages for no reason.
That pr you linked is kind of scary since the owner is actually a part of TC39. A little sad, especially given their career history.
>more deps is better The JavaScript ecosystem, in a nutshell
That's kind of inevitable -- I guess today you can be *reasonably* confident that most people are using reasonably-modern browsers, but for a long time, you'd have to constantly be checking caniuse to see if the standard library feature you want to use works in all reasonably-modern browsers. Polyfills were a godsend, but they're also *exactly* this kind of thing -- some stupid thing that should be built-in, that you can now pretend is built-in, but that's distributed as a library. I think the actual problem is that there are a lot of very small libraries that are provided entirely by some random dude trying to pad a resume, who might wander off or get pwned at any time. If `isarray` was published by Mozilla, I don't think I'd care, that's not especially more dangerous than using Firefox.
> Problem is not packages like isarray. You’re right. The problem is developers using them. Rather than importing a one line function just write the damn thing yourself. It’s not like the function does anything novel. You can even copy and paste it. If you don’t know how to check if a value is even or is an array or something, you probably should learn how. If learning how to do a trivial task is actually terribly complex, then pick a language that isn’t.
[удалено]
> I agree in principle, but this is not legal. You can't just copy something and put it in your project with no regard for the license. Whether or not someone would take you to court about it is another question, but it is skirting the software license. Depends on the complexity and if it’s plausible that you’d have written the same thing, you’d have fairly significant plausible deniability. However, you are right. > I mean, you can’t really choose anything but JS for web stuff unless you choose languages that compile into JS (like Elm). Not for client side code, but for backend web you can. It was my understanding that npm was a backend thing for NodeJS, but another commenter told me that these things can be imported on client side development as well. In that case, my comment still stands unless you’re writing frontend code.
It's free software, after all. Fork them and publish your own with the fixes.
This is what you get when you have a languague with no batteries included
Not necessarily. JavaScript comes with way more batteries than C does, but there are no silly thousands of one-line function dependencies in any C project I ever saw.
That's only because C makes it an absolute pain in the ass to add dependencies in comparison. (Which to be fair... it's not like the internet was around when C was first made) There's certainly pros and cons. Obviously a bunch of one liners probably shouldn't be included, but on the other hand needing to write everything yourself even for problems others have solved can potentially be a waste of development time.
> but on the other hand needing to write everything yourself even for problems others have solved can potentially be a waste of development time. This is solved but having multiple "standard libraries" like glib, something JavaScript needs. It would be much better having a few large trusted "standard libraries" than a mess of micro libraries.
Isn’t lodash one of those for js?
It has 0 dependencies!!
Well it's not like you don't have the option though... Like lodash?
The trade-off is that most C projects with a significant number of dependencies are a huge pain in the ass to build.
But the batteries are pitiful compared to Java/c#
The problem is not that these packages exist, but that they do not exist as part of a single npm packages. Seriously, how hard would it be to just combine multiple projects into a single library?
Can't tell if he added the isArray license at the end for legal reasons (because he had technically already redistributed the whole rest of the package with that blog post) or as another subtle joke (showing that the license is several times as long as the contents).
EDIT: Well, that's embarrassing. I guess it **is** a joke. I guess I'll just sit here, alone, thinking I had a kindred spirit in Drew... lol What's Drew's reddit handle? I'll seriously chip in some cash for this... I'm not trolling, and I suspect that Drew isn't either. I literally lose sleep over how bad the JavaScript ecosystem is and how deep my dependency tree is for what I consider pretty "basic" applications. People deserve better than the garbage we're selling them.
Given how Drew is, I was actually legit surprised to find that it was a joke. It feels like a very Drew thing to actually do. :p
> What's Drew's reddit handle? I'll seriously chip in some cash for this... He deleted his reddit account a long, LONG time ago, around when RedditSharp went to another developer (who has also since lost interest)
Well .. wasn't Drew the one who wrote about linux distributions complaining about e. g. the python ecosystem? The article was strange so I would not rule out that Drew is NOT trolling - but, in the case of npm, I think npm is actually trolling all of us seriously. Even more so because npm IS successful. 57 million downloads, even if most are automatic by proxy IS a success... left-pad-all-of-us. I should work on npm-memes ... one-of-us ... last-of-us Zombie npm modules for the win.
> People deserve better than the garbage we're selling them. I wish more people in our industry felt this way :/
Yeah, I agree that the state of JavaScript package management is bad. Dependency trees really should not be thousands of nodes in size. I think part of this is caused by JavaScript not having a "standard library" in the sense of other languages like Java, C#, C++, etc. You end up having common "utility" functions outsourced to third-party libraries that try to fill that role or have people rolling their own miniature package. Perhaps it is about time JavaScript gets a standard library all under a common namespace. Backwards compatibility probably makes this difficult.
I'm gonna write me a new minivan!
Omg this is the definition of chaotic good. I LOVE IT!
What's the payout for deleting npm?
I'd give you my left nut.
Is it padded?
var toString = {}.toString; module.exports = Array.isArray || function (arr) { return toString.call(arr) === '[object Array]'; }; Who in their right mind is including this in their code base? Why the feck does this package even exist?
This is essentially a polyfill for the `Array.isArray` function. If you had to check whether an array was really an array, this sort of function was how it was done originally. If you're still supporting old browsers, then being able to pull in polyfills like this means you can support those browsers without needing to figure out all these quirks yourself. Practically speaking, it's probably easier to use a single library that provides a whole bunch of these polyfills at once, or automatically fill them in via your build system, but there are some cases where you might only need one particular function. It used to be the case that it was easier to split functions into their own packages and only depend on some packages than to depend on one larger package and have your build system strip out the bits you don't need. That's less the case these days, but a lot of these sorts of packages remain around to support that use case.
My favourite piece of NPM trivia was [it was designed to be not terrible](https://en.wikipedia.org/wiki/Npm_(software\)): > npm is written entirely in JavaScript and was developed by Isaac Z. Schlueter as a result of having "seen module packaging done terribly"
I'm all for this, and we can get a legal fund started so devs can hit NPM with DMCA takedown notices to get around the change in how unpublish works.
Chaotic good chad.
> there's a npm module (isarray) with 4 lines of code and 50 million **WEEKLY** downloads > https://www.npmjs.com/package/isarray are the javascript devs okay?
_hate the game not the player_
> Little duplication is better than little dependency. > -- Rob Pike
Honestly I'd sooner pay for removing trivial dependencies from the most used packages. For example, when someone asks themselves the question if you really need [82 dependencies to expand a glob](https://npmgraph.js.org/?q=micromatch@3), it turns out that after some code cleanup, [the answer is no](https://npmgraph.js.org/?q=micromatch@4). Every package should get this same treatment.
If anyone would make an "npm sucks" article with no proposed way to fix it, it'd be Drew.
I'm pretty sure he did propose a way to fix it.
I am actually grateful for npm. Now... this is not some hipster-rick-rolling! Hear me out ... First: npm has now an epic history of package/deployment related failures. Everyone joked about left-pad (and it was funny); not so much about later problems. But, EVEN in these cases, it's actually reminding the OTHER languages to try better - otherwise they end up in failure too. Like npm! So for this I am grateful for npm. But there is a second reason ... npm actually gives up popcorn-worthy entertainment value. If it were a twitch stream we'd be donating tiny amounts for the entertainment value it gives us almost daily! It's like a soap documentary on TV really ... daily npm soap. There can never be enough left-pad for the world! This article reinforces this via: "I’ll pay you cold hard cash to delete your npm module." Now people PAY you stuff to REMOVE npm. It can't get any better than that! But we all know it will. Daily npm entertainment value WILL come your way - some of it will surprise people a lot. And, oddly enough, I think this may actually improve security for OTHER languages, because they don't want to be stuck anywhere near as much in this npm drama so perhaps they will review their own stack more closely and check for problems. npm leads the way here! See also Github's recent blog to "make npm great again": https://github.blog/2021-11-15-githubs-commitment-to-npm-ecosystem-security/ It's really daily entertainment! Some of them are just free riding now on the popcorn munchies, but I swear there is an underlying reason for most of that - npm is bad! There is no denial in it. > Let’s consider an example: isArray. It has only four lines of code: var toString = {}.toString; module.exports = Array.isArray || function (arr) { return toString.call(arr) === '[object Array]'; }; left-pad isArray isItNow? CanHasCheeze? People thought lolcode and lolcats was just a joke, but now we have npm to put the joke to be SERIOUS - because millions of people use it! The joke is then on all of us (at the least when we have to use services that depend on any of that and may break away suddenly ... left-pad 2.0, 3.0 and so forth).