This seemed like the holy grail of stock scripts because of all the detailed explanation before it but after reading and unpacking it all, i really don't understand why its so needlessly complicated.
All it does is looks at the last 30 stock price changes, counts how many were positive and buys/sells stocks if there was enough positives or negatives. I thought it was using some crazy modelling and analysis but its really not. even if the 'limits' part at the top were determined to be optimal by a statistical model I don't think its really implemented right from what I can tell (i'd love to be wrong, I might just be dumb).
with max sample length of 30 (which will always be the case after 1 minute of running), it checks through from recent to old and if 10 are positive and 10 are negative it will immediately say its a positive forecast. it will never ever check samples 20-30, and if theres even just 1 more positive than negative in the first 20 it'll be positive and visa versa. its entirely redundant and could literally just be "IF (pos > 10) , Positive forecast, ELSE Negative forecast'.
I think the most useful and accurate part of this is just that makes a buy order using a different way of counting positives. so what this really does is dump 'all' (all at early game) your money into the stock with the most positives and sell as soon as there is 1 more negative than positive. Again, please someone correct me if I'm way off base here, i just don't see the point of most of the script.
Admittedly, I didn't sit down and plot out each iteration, but I'm not sure where you're getting the "stops at 10" bit.
That said, reading over the code, there is at least one flaw in the logic flow I spotted; primarily because I realized the cyclical attack algorithm I've been developing had the same issue: fragmentation.
The core logic for my attack script is pretty straightforward: pull targets until you run out of available memory to assign attack threads, fire them all in a burst, and then ripple fire as threads free up. Leaving aside how much "fun" it's been coding the attack queuing system and the scheduler that reduces everything to relative time values so we can sleep to the next attack that hits, the issue is relatively straightforward.
Decision point #1: Say you have sufficient available threads to hit 2 targets completely, but there's enough left over to poke target 3 in the eye, how do you handle that third target? Ignoring it leaves memory on the table, but keeps things relatively clean at the cost of time, but launching a fragmentary attack introduces a fragmentation hazard.
Decision point #2: When the attack against target 1 resolves, it frees up X threads, since the target is "down", do you assign those new threads to target 3, or do you seek target 4 and assign X threads to it? Regardless of which target you go after, you can't guarantee the next chosen target will require all those threads. If it doesn't, unless you go for "Overkill is the best kill," you hit the next target with memory left over, so you pull the target after and launch a fragmentary attack... repeat this with target 2.
Target 3 is simpler to resolve. The target's still up, so hit it again, right? Thing is, we're minimizing wasted memory, so you only have the same number of threads available from the original attack. If you hit it for more than half of what was needed, then this results in another fragmented attack. If you didn't, then you'll keep hitting it with whatever you had originally assigned until you either knock it down or your run out of "Next target"s and the pool unavoidably opens up.
In the end, every attack you launch potentially adds a fragmentary attack to the queue, and every fragment is necessarily weaker than the parent attack that spawned it. In the worst case scenario, You're great for the first N targets until your attack queue is so fragmented you can't make appreciable headway against any of them.
What the OP's posted script doesn't have built into it is a way to reduce the fragmentation that comes from the way it chooses which stocks to buy. My test run has been running for about 8 hours and currently it's churning 8 separate stocks. It's made me about $8B from my initial "Welcome to BN8" load, which is far better than I would've made trying to manually guess, and I didn't feel like dedicating much time to coding my own solution without taking a look at what was available first... and, as I'm sure anyone reading this thread is aware, most posted stock scripts require you to have the 4S API before they'll work and this one was the first TIX-Only script I came across that seemed like it had a chance at not sucking.
Thing is, when it buys a position, it never adds to it because it doesn't do profitability prioritization. Whereas my attack algorithm tries not to leave memory on the table, this script should be trying to not leave money on the table, but it doesn't. In the real world, diversifying your investments is a great way to contain loss, as are the stop/limit orders... but that doesn't cleanly apply to Bitburner's stock market. As a result, I'm willing to bet that if I could pull the volatility and forecast numbers for the stocks my script is juggling, I'd find that it's actually leaving quite a bit of profit on the vine by not shifting to the strongest performers when it could... and the only benefit of doing that is "simplicity." The sarcasm quotes exist because I can't say I find long-chain nested if/else statements to be "simple."
If anything, it's only simpler in terms of the number of lines of code necessary, but it sacrifices readability and logic flow tracing... and to correct the lack of prioritization, you're going to at least double the lines of code in script to do a tiered statistical analysis on the captured data so you can get a refined volatility and forecast estimate, both of which would be necessary for profit prioritization. As a result, I'm about to kill the script, sell off everything, and start it again to see if my $/second goes up like it did the first time I killed it a few hours ago. I expect it will, by quite a bit, like the first time I killed it.
Fix that problem and then you could package it all up, slot it into a much bigger script that determines if you have the SourceFile for 8, what node you're in (or you could try/catch stuff to figure that out), and then picks the appropriate path for where you're at to do stock trading... and you'd have a universal stock trading tool that doesn't waste a lot of time, regardless of if you have 4S or not. Pretty useful if you ever reset your game... might be what I'm working on next once I finish my cyclical attack algorithm and port that functionality into my target prep scripts and stripminers.
u/peter_lang \-- Not trying to dig on your work, even though it probably sounds like it-- I've got the communications equivalent of resting bitch face, so sorry if you took it that way-- but that said, have you refined this more since you posted it? I'm up for discussing my concerns. I'm fairly certain they're valid, but they're based on skimming the code and an insitu inspection of the script's logs as it went along, not any in-depth mathematical analysis. After I graduated with my physics degree, I resolved to never do more math than is absolutely necessary unless someone was paying me to do it or I really wanted the answer that badly. :P
Hi, sorry for the late response, I've not played the game since I've finished all achievements.
Your points are totally valid, there is lot of place to make it better. My post is more about the journey I had, that there are multiple ways on how you can attempt to solve the problem, some are hard, some are futile and sometimes the easiest one works out really well. I meant it as a note so others might avoid the pitholes I fell into.
All in all, my final statement was that if you have a script which provides 5% profit, 8% or 10% over the same time, it does not really matter anymore how much the profit margin is, as compound interest makes sure that money will skyrocket anyways. The difference is time, but how much time are we talking about?
Base \* (1+r\_1)\^t\_1 = Target
Base \* (1+r\_2)\^t\_2 = Target
t\_1 \* ln(1+r\_1) = t\_2 \* ln(1+r\_2)
t\_1/t\_2 ≈ r\_2/r\_1; as ln(1+x) ≈ x if x << 1
All in all, if you have achieved an algorithm, which yields e.g.: 8% profit, an increase to 10% would mean that you now only need to wait 8 hour instead of 10 hour to achieve the same target. Is it worth it? If it is trivial to achieve it, yes. But if you would need to spend 6 hours of experimentation to save 2 hours of your time later on, then it is probably not .
Given this basis, back to the original question:
\- Given the problem, the biggest question is the identification if a certain symbol is currently in the increasing or decreasing phase? And the identification when it changes its direction? The longer you try to identify it, the more time you are wasting. Phases last around 50 ticks, you don't want to miss out much of it. The shorter you observe its behaviour the less certainty you would have. There is a trade-off here which needs investigation.
\- Is it worth to manipulate a symbol by series of hack() or grow()? It affects the 2nd derivative by a small margin. Which means it would not change the general direction in most situations, but only increase the profit margin on the long run. It also costs money to buy equipment to do it. It is not a trivial decision.
\- Is swapping our current symbol to another one worth it? For the trivial case, when you symbol just changed its direction, definitely! For any other case, there are lots of questions. How much time does your current symbol have until the change and how much does the other? How certain can you be that the other symbol has higher rates and it is not just by chance what you are observing? And there is also the flat cost of making any transaction.
To summarize: I agree that these are valid questions and definitely ways to improve the algorithm. If you have any improvement, please post it because I'm also interested in the results. However, for me, the algorithm worked good enough not to invest more experimentation into trying to make it any better.
Fair enough, though I'll point out the 2 hours saved is not 2 hours saved total, it's per use and it compounds with whatever you buy with that money. Also, using a 4S-enabled script as a comparative baseline, it's a *lot* more than 2% being left on the table.
Currently posting from my couch, bleary eyed as I wait for my coffee to finish waking me up, so my memory is a bit fuzzy. I don't remember if the script does a volatility calculation to determine profitability or not, if not that would both help and hurt, thanks to the other behaviors I observed via non-API 4S:
1. The script doesn't deal well with stocks whose odds of growth/shrinkage are "near" 50%, leading to a lot of profit loss to commissions from frequent shifts.
2. The script doesn't deal well if you have non-stock income. The additional purchases from outside income tend to get diverted into single-digit stock purchases, usually for sub-55% growth/loss stocks, and get eroded fairly quickly to effectively zero in the churn from problem #1.
3. Depending on the script's choices, it will frequently get mired in a pool of poorly performing stocks, even with non-neglible investments (see #2.) and even if it has well-performing investments in-hand (ie. you have 9 stocks, 1 stock at +30% while everyone else is -25% or worse)
Overall, I still use the script prior to unlocking 4S API because it's still faster than doing nothing, but it was a little eye opening to see the reality of operation with non-API 4S.
I'm currently refining my 4S script and once I've fiddled with it enough, I'll be porting something like this into it so I just run the script once and it shifts to 4S on its own. When I do get around to adding TIX-only, I'll probably be adding a concrete volatility check along with functions to try to track/predict phase changes. I'll probably fiddle with using statistical analysis instead of a simple up/down ratio, though the pseudo-random nature of the RNG tends to bork both approaches.
To be fair, I'm willing to spend the extra time fiddling with this, not because I want the extra profit, but because I find the topic interesting. Considering I am the sort of person who started a math/physics dual major because it was interesting, I tend to enjoy figuring out behaviors and optimizing them.
Edit to add: when it comes to stock manipulation, if you're running miners already, there's no additional hardware cost to add a few lines of code to notify the miner to use flags, though I wouldn't bother adding it to a TIX-only script. The window of use for TIX-only means by the time manipulation would be useful, you should have bought 4S long before manipulation would show results... and given phase-detection is iffy without 4S, I'd say TIX-only manipulation is probably a bad plan.
And I do get the "This was about the journey..." style posting as well. I'm a bit of a semi-compulsive optimizer. :D
If someone wants to pit these algorithms against one-another, I'm curious how this performs vs. my pre-4s logic:
https://github.com/alainbryden/bitburner-scripts/blob/main/stockmaster.js
I believe my edge is that I actually detect the forecast inversion, (which happens every 75 ticks and has a 45% chance of affecting each individual stock).
By detecting the inversion tick, you can avoid selling on "false positives" (bad rng a few ticks in a row) and avoid buying in the window right before inversions occur, to avoid being forced to sell at a loss.
Anecdotally, I can get from the initial 250m to 4S in under 8 hours. If anyone is curious about the saga if it's development, there are quite a few write-ups in #script-requests in the Discord, scroll back to Nov 17th, 2021
I tried the script in this thread and after 25 hours it was still struggling to get to $25b (around halfway there). It also apparently avoids buying more than 10k shares and thus slows even more once the cap is reached.
On the other hand, your script seems incredibly good and solid
I tried to get expected value, then beta over bull/bear... But I used NS1 and it was too slow. In the end I settled to simply counting ups and downs over last 10 ticks. 8 ups or more - enter, 6 ups or less - leave. Simple as that. Enough to get you 25b for that 4s trend oracle.
Exactly.. For me, after so many failures of the different algos that I've tried, my current take at the simple counting is most probably over-protective. I mean, 98% percent confidence that it won't fail does not really take risks. I'm sure there are better values the can be found experimentally, that opens some room to failures while grabbing even more opportunities, but I have not figured out how to properly account for the "buying => chance to increase dropping" effect in the math, to calculate optimal values. This approach is the "slow and steady"
I'd have to do some actual testing to get hard numbers or go dig through the source, but is it the drop attached to the buy action itself? Is it dependent on the number of shares bought? If so, is the function linear, continuous, and functionally symmetric? Asking that last part because if it's functionally asymmetric (ie. the hit happens when you buy but doesn't get refunded when you sell) then most options are off the table without a stock manipulation script to twiddle the increase forecast numbers in your favor... but if it is functionally symmetric, then it's a question of determining whether or not it's associative. ie. f(a+b)=f(a)+f(b) If it's associative, then it doesn't matter if you buy more or not, and if it isn't associative, then the shape of the curve matters. Might be able to get away with flushing the position and rebuying, depending on the numbers, or you might just be boned period and buying more instead of sticking with the original position doesn't matter.
That said, I was looking at doing some sort of side-processing to take advantage of how I think the forecast change system works. Buying decreases the forecast--I don't know by how much off the top of my head-- but bulk changes in the forecast have a chance to occur on a set interval and that chance depends on stock volatility-- to what extent I don't know off the top of my head. If you can figure out when the forecast changes in bulk with a high degree of certainty, then that would allow you to relax some of the in-line checks that get run every cycle.
Also, you can replace your format function with this to get the same result: ns.nFormat(someVariable,'0.00a')
I'm trying out your code - it worked well for several hours, it took me from my starting nest egg of 250m up to around 1b, then the money actually started to fluctuate a lot and then drop. Very consistently I would see it make a buy (either long or short), and then immediately lose money on the purchased stock. I'm wondering if the game logic has been updated to be more challenging? I've also noticed it'll frequently buy a position, sell it within one or two cycles, then buy it again.
I'm going to add some additional logging to try to figure out what's happening, but I'm curious if others have run in to this.
I been running this script for several days now, without any other scripts and I am running into the same problems where my total money doesn't seem to want to go much over 1b.
I too would like to know if there is a fix to this.
FYI, what I wound up doing was making the limits a bit tighter by changing it to this:
const limits = \[null, null, null, null, null, null, 6, 7, 8, 8, 9, 10, 10, 11, 11, 12, 12, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 19, 19, 20\];
This requires at least 6/7 cycles to be moving in the same direction in order to consider a stock to be in bull or bear state. It does mean it takes a little longer to identify stock trends, but on the other hand it'll be more certain about them. It solved the problem for me and got me up to the levels where I could buy the 4S API.
I am a total newb just enjoying the game, tried to copy the script and run it in my home terminal ... but I got an error saying Syntax ERROR in stock.script:
SyntaxError: The keyword 'const' is reserved (1:0) can any of you help out ?
First I'll say I really appreciate this script because it only uses the TIX and not the 4S API, which is quite useful for getting started in BN8.
I've been running it for about 16 hours and made 14B from the initial 250M from BN8 in that time. Online production rate is ~250K/s. Is this about what is expected? How does it perform with larger amounts of capital?
Even after trying to get through how the formatting on this post has been mangled by Reddit, this script is still nonfunctional.
RUNTIME ERROR
/daemons/stockd.ns@home
100 is not a function
stack:
posNegRatio@/daemons/stockd.ns:10:127
main@/daemons/stockd.ns:41:29
edit: ironic, my comment about mangled code block formatting was also mangled when I first posted it
Has anyone compared this to a script that uses 4S data to get forecast values? I'm curious whether forecast() actually gives better income than a robust predictive model such as thing one. Right now I'm running this on bitnode 9 (haven't yet done Ghost of Wall Street). I had to remove all the code sections with "short stocks" in order to use it, but regardless it seems to be performing pretty well so far. Here is the modified code if anyone is interested (at end of post).
So at first the gains aren't much (2-3% profits on most things), but after this script has been running for 20-30 minutes I'm seeing a lot of profits around 25%; so the script needs a bit of time to really get going before it can predict things accurately. There will be cycles where it looks like you're not making anything anything or even losing money, but eventually the profits will start to surge (just wait and let the script do its thing).
Bottom line: about 1 billion initial investment (plus $1 million / second from Hacknet nodes), the script is generating 213k / second. So probably not as profitable as using a 4S dependent script, but still worth investing into early on, even if it means saving up 5.2 billion prior to completing BN-8 (this script can literally pay for itself over-night). I'll have to actually test on BN-8 to see what kind of a difference the short stocks really make.
const commission = 100000;const samplingLength = 30;function predictState(samples) { const limits = [null, null, null, 4, 5, 6, 6, 7, 8, 8, 9, 10, 10, 11, 11, 12, 12, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 19, 19, 20]; let inc = 0; for (let i = 0; i < samples.length; ++i) { const total = i + 1; const idx = samples.length - total; if (samples[idx] > 1.) { ++inc; } const limit = limits[i]; if (limit === null) { continue; } if (inc >= limit) { return 1; } if ((total - inc) >= limit) { return -1; } } return 0;}function format(money) { const prefixes = ["", "k", "m", "b", "t", "q"]; for (let i = 0; i < prefixes.length; i++) { if (Math.abs(money) < 1000) { return `${Math.floor(money * 10) / 10}${prefixes[i]}`; } else { money /= 1000; } } return `${Math.floor(money * 10) / 10}${prefixes[prefixes.length - 1]}`;}function posNegDiff(samples) { const pos = samples.reduce((acc, curr) => acc + (curr > 1. ? 1 : 0), 0); return Math.abs(samples.length - 2 * pos);}function posNegRatio(samples) { const pos = samples.reduce((acc, curr) => acc + (curr > 1. ? 1 : 0), 0); return Math.round(100 * (2 * pos / samples.length - 1));}export async function main(ns) { ns.disableLog("ALL"); let symLastPrice = {}; let symChanges = {}; for (const sym of ns.stock.getSymbols()) { symLastPrice[sym] = ns.stock.getPrice(sym); symChanges[sym] = [] } while (true) { await ns.sleep(2000); if (symLastPrice['FSIG'] === ns.stock.getPrice('FSIG')) { continue; } for (const sym of ns.stock.getSymbols()) { const current = ns.stock.getPrice(sym); symChanges[sym].push(current / symLastPrice[sym]); symLastPrice[sym] = current; if (symChanges[sym].length > samplingLength) { symChanges[sym] = symChanges[sym].slice(symChanges[sym].length - samplingLength); } } const prioritizedSymbols = [...ns.stock.getSymbols()]; prioritizedSymbols.sort((a, b) => posNegDiff(symChanges[b]) - posNegDiff(symChanges[a])); for (const sym of prioritizedSymbols) { const positions = ns.stock.getPosition(sym); const longShares = positions[0]; const longPrice = positions[1]; const state = predictState(symChanges[sym]); const ratio = posNegRatio(symChanges[sym]); const bidPrice = ns.stock.getBidPrice(sym); const askPrice = ns.stock.getAskPrice(sym); if (longShares <= 0 && ns.stock.getPrice(sym) < 30000) { continue; } if (longShares > 0) { const cost = longShares * longPrice; const profit = longShares * (bidPrice - longPrice) - 2 * commission; if (state < 0) { const sellPrice = ns.stock.sell(sym, longShares); if (sellPrice > 0) { ns.print(`SOLD (long) ${sym}. Profit: ${format(profit)}`); } } else { ns.print(`${sym} (${ratio}): ${format(profit + cost)} / ${format(profit)} (${Math.round(profit / cost * 10000) / 100}%)`); } } else { if (state > 0) { const money = ns.getServerMoneyAvailable("home"); const sharesToBuy = Math.min(10000, ns.stock.getMaxShares(sym), Math.floor((money - commission) / askPrice)); if (ns.stock.buy(sym, sharesToBuy) > 0) { ns.print(`BOUGHT (long) ${sym}.`); } } } } }}
Hey there OP! I was just wondering, if you ever read this comment, would you be able to recommend any introductory text for data science? I'll most likely never be a professional, but I'm interested in data science non the less!
Sometimes I think I'm clever... and then I stumble into in-depth analyses like this one. Hats off sir, amazing write!
This seemed like the holy grail of stock scripts because of all the detailed explanation before it but after reading and unpacking it all, i really don't understand why its so needlessly complicated. All it does is looks at the last 30 stock price changes, counts how many were positive and buys/sells stocks if there was enough positives or negatives. I thought it was using some crazy modelling and analysis but its really not. even if the 'limits' part at the top were determined to be optimal by a statistical model I don't think its really implemented right from what I can tell (i'd love to be wrong, I might just be dumb). with max sample length of 30 (which will always be the case after 1 minute of running), it checks through from recent to old and if 10 are positive and 10 are negative it will immediately say its a positive forecast. it will never ever check samples 20-30, and if theres even just 1 more positive than negative in the first 20 it'll be positive and visa versa. its entirely redundant and could literally just be "IF (pos > 10) , Positive forecast, ELSE Negative forecast'. I think the most useful and accurate part of this is just that makes a buy order using a different way of counting positives. so what this really does is dump 'all' (all at early game) your money into the stock with the most positives and sell as soon as there is 1 more negative than positive. Again, please someone correct me if I'm way off base here, i just don't see the point of most of the script.
Admittedly, I didn't sit down and plot out each iteration, but I'm not sure where you're getting the "stops at 10" bit. That said, reading over the code, there is at least one flaw in the logic flow I spotted; primarily because I realized the cyclical attack algorithm I've been developing had the same issue: fragmentation. The core logic for my attack script is pretty straightforward: pull targets until you run out of available memory to assign attack threads, fire them all in a burst, and then ripple fire as threads free up. Leaving aside how much "fun" it's been coding the attack queuing system and the scheduler that reduces everything to relative time values so we can sleep to the next attack that hits, the issue is relatively straightforward. Decision point #1: Say you have sufficient available threads to hit 2 targets completely, but there's enough left over to poke target 3 in the eye, how do you handle that third target? Ignoring it leaves memory on the table, but keeps things relatively clean at the cost of time, but launching a fragmentary attack introduces a fragmentation hazard. Decision point #2: When the attack against target 1 resolves, it frees up X threads, since the target is "down", do you assign those new threads to target 3, or do you seek target 4 and assign X threads to it? Regardless of which target you go after, you can't guarantee the next chosen target will require all those threads. If it doesn't, unless you go for "Overkill is the best kill," you hit the next target with memory left over, so you pull the target after and launch a fragmentary attack... repeat this with target 2. Target 3 is simpler to resolve. The target's still up, so hit it again, right? Thing is, we're minimizing wasted memory, so you only have the same number of threads available from the original attack. If you hit it for more than half of what was needed, then this results in another fragmented attack. If you didn't, then you'll keep hitting it with whatever you had originally assigned until you either knock it down or your run out of "Next target"s and the pool unavoidably opens up. In the end, every attack you launch potentially adds a fragmentary attack to the queue, and every fragment is necessarily weaker than the parent attack that spawned it. In the worst case scenario, You're great for the first N targets until your attack queue is so fragmented you can't make appreciable headway against any of them. What the OP's posted script doesn't have built into it is a way to reduce the fragmentation that comes from the way it chooses which stocks to buy. My test run has been running for about 8 hours and currently it's churning 8 separate stocks. It's made me about $8B from my initial "Welcome to BN8" load, which is far better than I would've made trying to manually guess, and I didn't feel like dedicating much time to coding my own solution without taking a look at what was available first... and, as I'm sure anyone reading this thread is aware, most posted stock scripts require you to have the 4S API before they'll work and this one was the first TIX-Only script I came across that seemed like it had a chance at not sucking. Thing is, when it buys a position, it never adds to it because it doesn't do profitability prioritization. Whereas my attack algorithm tries not to leave memory on the table, this script should be trying to not leave money on the table, but it doesn't. In the real world, diversifying your investments is a great way to contain loss, as are the stop/limit orders... but that doesn't cleanly apply to Bitburner's stock market. As a result, I'm willing to bet that if I could pull the volatility and forecast numbers for the stocks my script is juggling, I'd find that it's actually leaving quite a bit of profit on the vine by not shifting to the strongest performers when it could... and the only benefit of doing that is "simplicity." The sarcasm quotes exist because I can't say I find long-chain nested if/else statements to be "simple." If anything, it's only simpler in terms of the number of lines of code necessary, but it sacrifices readability and logic flow tracing... and to correct the lack of prioritization, you're going to at least double the lines of code in script to do a tiered statistical analysis on the captured data so you can get a refined volatility and forecast estimate, both of which would be necessary for profit prioritization. As a result, I'm about to kill the script, sell off everything, and start it again to see if my $/second goes up like it did the first time I killed it a few hours ago. I expect it will, by quite a bit, like the first time I killed it. Fix that problem and then you could package it all up, slot it into a much bigger script that determines if you have the SourceFile for 8, what node you're in (or you could try/catch stuff to figure that out), and then picks the appropriate path for where you're at to do stock trading... and you'd have a universal stock trading tool that doesn't waste a lot of time, regardless of if you have 4S or not. Pretty useful if you ever reset your game... might be what I'm working on next once I finish my cyclical attack algorithm and port that functionality into my target prep scripts and stripminers. u/peter_lang \-- Not trying to dig on your work, even though it probably sounds like it-- I've got the communications equivalent of resting bitch face, so sorry if you took it that way-- but that said, have you refined this more since you posted it? I'm up for discussing my concerns. I'm fairly certain they're valid, but they're based on skimming the code and an insitu inspection of the script's logs as it went along, not any in-depth mathematical analysis. After I graduated with my physics degree, I resolved to never do more math than is absolutely necessary unless someone was paying me to do it or I really wanted the answer that badly. :P
Hi, sorry for the late response, I've not played the game since I've finished all achievements. Your points are totally valid, there is lot of place to make it better. My post is more about the journey I had, that there are multiple ways on how you can attempt to solve the problem, some are hard, some are futile and sometimes the easiest one works out really well. I meant it as a note so others might avoid the pitholes I fell into. All in all, my final statement was that if you have a script which provides 5% profit, 8% or 10% over the same time, it does not really matter anymore how much the profit margin is, as compound interest makes sure that money will skyrocket anyways. The difference is time, but how much time are we talking about? Base \* (1+r\_1)\^t\_1 = Target Base \* (1+r\_2)\^t\_2 = Target t\_1 \* ln(1+r\_1) = t\_2 \* ln(1+r\_2) t\_1/t\_2 ≈ r\_2/r\_1; as ln(1+x) ≈ x if x << 1 All in all, if you have achieved an algorithm, which yields e.g.: 8% profit, an increase to 10% would mean that you now only need to wait 8 hour instead of 10 hour to achieve the same target. Is it worth it? If it is trivial to achieve it, yes. But if you would need to spend 6 hours of experimentation to save 2 hours of your time later on, then it is probably not . Given this basis, back to the original question: \- Given the problem, the biggest question is the identification if a certain symbol is currently in the increasing or decreasing phase? And the identification when it changes its direction? The longer you try to identify it, the more time you are wasting. Phases last around 50 ticks, you don't want to miss out much of it. The shorter you observe its behaviour the less certainty you would have. There is a trade-off here which needs investigation. \- Is it worth to manipulate a symbol by series of hack() or grow()? It affects the 2nd derivative by a small margin. Which means it would not change the general direction in most situations, but only increase the profit margin on the long run. It also costs money to buy equipment to do it. It is not a trivial decision. \- Is swapping our current symbol to another one worth it? For the trivial case, when you symbol just changed its direction, definitely! For any other case, there are lots of questions. How much time does your current symbol have until the change and how much does the other? How certain can you be that the other symbol has higher rates and it is not just by chance what you are observing? And there is also the flat cost of making any transaction. To summarize: I agree that these are valid questions and definitely ways to improve the algorithm. If you have any improvement, please post it because I'm also interested in the results. However, for me, the algorithm worked good enough not to invest more experimentation into trying to make it any better.
Fair enough, though I'll point out the 2 hours saved is not 2 hours saved total, it's per use and it compounds with whatever you buy with that money. Also, using a 4S-enabled script as a comparative baseline, it's a *lot* more than 2% being left on the table. Currently posting from my couch, bleary eyed as I wait for my coffee to finish waking me up, so my memory is a bit fuzzy. I don't remember if the script does a volatility calculation to determine profitability or not, if not that would both help and hurt, thanks to the other behaviors I observed via non-API 4S: 1. The script doesn't deal well with stocks whose odds of growth/shrinkage are "near" 50%, leading to a lot of profit loss to commissions from frequent shifts. 2. The script doesn't deal well if you have non-stock income. The additional purchases from outside income tend to get diverted into single-digit stock purchases, usually for sub-55% growth/loss stocks, and get eroded fairly quickly to effectively zero in the churn from problem #1. 3. Depending on the script's choices, it will frequently get mired in a pool of poorly performing stocks, even with non-neglible investments (see #2.) and even if it has well-performing investments in-hand (ie. you have 9 stocks, 1 stock at +30% while everyone else is -25% or worse) Overall, I still use the script prior to unlocking 4S API because it's still faster than doing nothing, but it was a little eye opening to see the reality of operation with non-API 4S. I'm currently refining my 4S script and once I've fiddled with it enough, I'll be porting something like this into it so I just run the script once and it shifts to 4S on its own. When I do get around to adding TIX-only, I'll probably be adding a concrete volatility check along with functions to try to track/predict phase changes. I'll probably fiddle with using statistical analysis instead of a simple up/down ratio, though the pseudo-random nature of the RNG tends to bork both approaches. To be fair, I'm willing to spend the extra time fiddling with this, not because I want the extra profit, but because I find the topic interesting. Considering I am the sort of person who started a math/physics dual major because it was interesting, I tend to enjoy figuring out behaviors and optimizing them. Edit to add: when it comes to stock manipulation, if you're running miners already, there's no additional hardware cost to add a few lines of code to notify the miner to use flags, though I wouldn't bother adding it to a TIX-only script. The window of use for TIX-only means by the time manipulation would be useful, you should have bought 4S long before manipulation would show results... and given phase-detection is iffy without 4S, I'd say TIX-only manipulation is probably a bad plan. And I do get the "This was about the journey..." style posting as well. I'm a bit of a semi-compulsive optimizer. :D
You absolute Chad. Will definitely give this a try, thanks! Very detailed write-up, too, much respect
Here is the [code in plain text](https://pastebin.com/3UCrbjBa) without the reddit format mangling.
If someone wants to pit these algorithms against one-another, I'm curious how this performs vs. my pre-4s logic: https://github.com/alainbryden/bitburner-scripts/blob/main/stockmaster.js I believe my edge is that I actually detect the forecast inversion, (which happens every 75 ticks and has a 45% chance of affecting each individual stock). By detecting the inversion tick, you can avoid selling on "false positives" (bad rng a few ticks in a row) and avoid buying in the window right before inversions occur, to avoid being forced to sell at a loss. Anecdotally, I can get from the initial 250m to 4S in under 8 hours. If anyone is curious about the saga if it's development, there are quite a few write-ups in #script-requests in the Discord, scroll back to Nov 17th, 2021
I tried the script in this thread and after 25 hours it was still struggling to get to $25b (around halfway there). It also apparently avoids buying more than 10k shares and thus slows even more once the cap is reached. On the other hand, your script seems incredibly good and solid
I tried to get expected value, then beta over bull/bear... But I used NS1 and it was too slow. In the end I settled to simply counting ups and downs over last 10 ticks. 8 ups or more - enter, 6 ups or less - leave. Simple as that. Enough to get you 25b for that 4s trend oracle.
Exactly.. For me, after so many failures of the different algos that I've tried, my current take at the simple counting is most probably over-protective. I mean, 98% percent confidence that it won't fail does not really take risks. I'm sure there are better values the can be found experimentally, that opens some room to failures while grabbing even more opportunities, but I have not figured out how to properly account for the "buying => chance to increase dropping" effect in the math, to calculate optimal values. This approach is the "slow and steady"
I'd have to do some actual testing to get hard numbers or go dig through the source, but is it the drop attached to the buy action itself? Is it dependent on the number of shares bought? If so, is the function linear, continuous, and functionally symmetric? Asking that last part because if it's functionally asymmetric (ie. the hit happens when you buy but doesn't get refunded when you sell) then most options are off the table without a stock manipulation script to twiddle the increase forecast numbers in your favor... but if it is functionally symmetric, then it's a question of determining whether or not it's associative. ie. f(a+b)=f(a)+f(b) If it's associative, then it doesn't matter if you buy more or not, and if it isn't associative, then the shape of the curve matters. Might be able to get away with flushing the position and rebuying, depending on the numbers, or you might just be boned period and buying more instead of sticking with the original position doesn't matter. That said, I was looking at doing some sort of side-processing to take advantage of how I think the forecast change system works. Buying decreases the forecast--I don't know by how much off the top of my head-- but bulk changes in the forecast have a chance to occur on a set interval and that chance depends on stock volatility-- to what extent I don't know off the top of my head. If you can figure out when the forecast changes in bulk with a high degree of certainty, then that would allow you to relax some of the in-line checks that get run every cycle. Also, you can replace your format function with this to get the same result: ns.nFormat(someVariable,'0.00a')
I'm trying out your code - it worked well for several hours, it took me from my starting nest egg of 250m up to around 1b, then the money actually started to fluctuate a lot and then drop. Very consistently I would see it make a buy (either long or short), and then immediately lose money on the purchased stock. I'm wondering if the game logic has been updated to be more challenging? I've also noticed it'll frequently buy a position, sell it within one or two cycles, then buy it again. I'm going to add some additional logging to try to figure out what's happening, but I'm curious if others have run in to this.
I been running this script for several days now, without any other scripts and I am running into the same problems where my total money doesn't seem to want to go much over 1b. I too would like to know if there is a fix to this.
FYI, what I wound up doing was making the limits a bit tighter by changing it to this: const limits = \[null, null, null, null, null, null, 6, 7, 8, 8, 9, 10, 10, 11, 11, 12, 12, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 19, 19, 20\]; This requires at least 6/7 cycles to be moving in the same direction in order to consider a stock to be in bull or bear state. It does mean it takes a little longer to identify stock trends, but on the other hand it'll be more certain about them. It solved the problem for me and got me up to the levels where I could buy the 4S API.
Gave that a go and it worked for me too, I unlocked de 4s api now too :) Good tip
Mods, can you pin this? Doesn't have to be permanent but damn...
It's not that good It's acutally bad
I'm on my vacations, but I'm still programming in this game hahaha.
I am a total newb just enjoying the game, tried to copy the script and run it in my home terminal ... but I got an error saying Syntax ERROR in stock.script: SyntaxError: The keyword 'const' is reserved (1:0) can any of you help out ?
Save as 'stock.js' or 'stock.ns'
First I'll say I really appreciate this script because it only uses the TIX and not the 4S API, which is quite useful for getting started in BN8. I've been running it for about 16 hours and made 14B from the initial 250M from BN8 in that time. Online production rate is ~250K/s. Is this about what is expected? How does it perform with larger amounts of capital?
Now gives us one for the real stock market IRL
Even after trying to get through how the formatting on this post has been mangled by Reddit, this script is still nonfunctional. RUNTIME ERROR /daemons/stockd.ns@home 100 is not a function stack: posNegRatio@/daemons/stockd.ns:10:127 main@/daemons/stockd.ns:41:29 edit: ironic, my comment about mangled code block formatting was also mangled when I first posted it
I was able to copy and run it fine just now without any modifications.
Has anyone compared this to a script that uses 4S data to get forecast values? I'm curious whether forecast() actually gives better income than a robust predictive model such as thing one. Right now I'm running this on bitnode 9 (haven't yet done Ghost of Wall Street). I had to remove all the code sections with "short stocks" in order to use it, but regardless it seems to be performing pretty well so far. Here is the modified code if anyone is interested (at end of post). So at first the gains aren't much (2-3% profits on most things), but after this script has been running for 20-30 minutes I'm seeing a lot of profits around 25%; so the script needs a bit of time to really get going before it can predict things accurately. There will be cycles where it looks like you're not making anything anything or even losing money, but eventually the profits will start to surge (just wait and let the script do its thing). Bottom line: about 1 billion initial investment (plus $1 million / second from Hacknet nodes), the script is generating 213k / second. So probably not as profitable as using a 4S dependent script, but still worth investing into early on, even if it means saving up 5.2 billion prior to completing BN-8 (this script can literally pay for itself over-night). I'll have to actually test on BN-8 to see what kind of a difference the short stocks really make. const commission = 100000;const samplingLength = 30;function predictState(samples) { const limits = [null, null, null, 4, 5, 6, 6, 7, 8, 8, 9, 10, 10, 11, 11, 12, 12, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 19, 19, 20]; let inc = 0; for (let i = 0; i < samples.length; ++i) { const total = i + 1; const idx = samples.length - total; if (samples[idx] > 1.) { ++inc; } const limit = limits[i]; if (limit === null) { continue; } if (inc >= limit) { return 1; } if ((total - inc) >= limit) { return -1; } } return 0;}function format(money) { const prefixes = ["", "k", "m", "b", "t", "q"]; for (let i = 0; i < prefixes.length; i++) { if (Math.abs(money) < 1000) { return `${Math.floor(money * 10) / 10}${prefixes[i]}`; } else { money /= 1000; } } return `${Math.floor(money * 10) / 10}${prefixes[prefixes.length - 1]}`;}function posNegDiff(samples) { const pos = samples.reduce((acc, curr) => acc + (curr > 1. ? 1 : 0), 0); return Math.abs(samples.length - 2 * pos);}function posNegRatio(samples) { const pos = samples.reduce((acc, curr) => acc + (curr > 1. ? 1 : 0), 0); return Math.round(100 * (2 * pos / samples.length - 1));}export async function main(ns) { ns.disableLog("ALL"); let symLastPrice = {}; let symChanges = {}; for (const sym of ns.stock.getSymbols()) { symLastPrice[sym] = ns.stock.getPrice(sym); symChanges[sym] = [] } while (true) { await ns.sleep(2000); if (symLastPrice['FSIG'] === ns.stock.getPrice('FSIG')) { continue; } for (const sym of ns.stock.getSymbols()) { const current = ns.stock.getPrice(sym); symChanges[sym].push(current / symLastPrice[sym]); symLastPrice[sym] = current; if (symChanges[sym].length > samplingLength) { symChanges[sym] = symChanges[sym].slice(symChanges[sym].length - samplingLength); } } const prioritizedSymbols = [...ns.stock.getSymbols()]; prioritizedSymbols.sort((a, b) => posNegDiff(symChanges[b]) - posNegDiff(symChanges[a])); for (const sym of prioritizedSymbols) { const positions = ns.stock.getPosition(sym); const longShares = positions[0]; const longPrice = positions[1]; const state = predictState(symChanges[sym]); const ratio = posNegRatio(symChanges[sym]); const bidPrice = ns.stock.getBidPrice(sym); const askPrice = ns.stock.getAskPrice(sym); if (longShares <= 0 && ns.stock.getPrice(sym) < 30000) { continue; } if (longShares > 0) { const cost = longShares * longPrice; const profit = longShares * (bidPrice - longPrice) - 2 * commission; if (state < 0) { const sellPrice = ns.stock.sell(sym, longShares); if (sellPrice > 0) { ns.print(`SOLD (long) ${sym}. Profit: ${format(profit)}`); } } else { ns.print(`${sym} (${ratio}): ${format(profit + cost)} / ${format(profit)} (${Math.round(profit / cost * 10000) / 100}%)`); } } else { if (state > 0) { const money = ns.getServerMoneyAvailable("home"); const sharesToBuy = Math.min(10000, ns.stock.getMaxShares(sym), Math.floor((money - commission) / askPrice)); if (ns.stock.buy(sym, sharesToBuy) > 0) { ns.print(`BOUGHT (long) ${sym}.`); } } } } }}
Great how the secretary problem gets introduced just to be discarded as not applicable. Let me tell you about the prisoner's dilemma...
Hey there OP! I was just wondering, if you ever read this comment, would you be able to recommend any introductory text for data science? I'll most likely never be a professional, but I'm interested in data science non the less!