[isatrader]

Orange and Blue combined would be a sum of stocks above the zero line and stocks with a rising zero line?  Perhaps a recalculation based on a score of +1 for RS above the zero line and zero line rising, -1 for RS below the zero line and zero line falling?  But I'd have to recalculate, which takes a long time.

No, the rising zero line wouldn't be an absolute requirement, it would be an OR calculation. i.e. if the % of stocks with Mansfield RS above zero OR if % of stocks with Mansfield RS above zero with rising zero line. As % of stocks with Mansfield RS above zero is the fast line and rising zero line is the slow line.

So get aggressive on orange above 50%, as it's highlighting a period where stocks have strong momentum. But tighten up stops and trim extended, sell laggards, but hold leaders on Blue falling back below 50% etc.

   

Swapping the index is easy.  I'll change the index to the S&P Small cap 600 later.  Don't seem to have the Russell 2000 available, except for a UK ETF which may have currency exchange rates overlaid - confusing the picture.

Not what you suggested, but swapping the underlying breadth pool is slower.  It was about 2 1/2 mins per point for these latest calcs.  My computer started to swap memory much more, so my plan to see if I can load all of the stocks in breadth pool to allow for MUCH faster historical breath calculations is probably a non-starter.  I can try loading the data to see if it is feasible fairly quickly though.

I'm only referring to visual referencing not changing the calculation.

On a separate note, with the development of the screener on the front end of the website I will have many years of Mansfield RS data for every US stock soon, and other markets potentially. UK, India etc. But starting with the US data. So i'll be able to do some backtests on the data.

[pcabc]

Orange and Blue combined would be a sum of stocks above the zero line and stocks with a rising zero line?  Perhaps a recalculation based on a score of +1 for RS above the zero line and zero line rising, -1 for RS below the zero line and zero line falling?  But I'd have to recalculate, which takes a long time.

No, the rising zero line wouldn't be an absolute requirement, it would be an OR calculation. i.e. if the % of stocks with Mansfield RS above zero OR if % of stocks with Mansfield RS above zero with rising zero line. As % of stocks with Mansfield RS above zero is the fast line and rising zero line is the slow line.

So get aggressive on orange above 50%, as it's highlighting a period where stocks have strong momentum. But tighten up stops and trim extended, sell laggards, but hold leaders on Blue falling back below 50% etc.

OK, see what you mean.  Basically enter aggressively on orange, ensure exit strategy is in place on drop in blue.  

Just a note, the blue line is dependent on the zero line rising or falling, whether the RS is above or below it is irrelevant.  I'm not sure if I got that across well.

I'm wondering if I could try a combined metric, a score and then produce an average score breadth plot?  Actually, no point wondering, I absolutely could.  Where there there is merit in doing so is the question?
 RS above zero line +2
 RS increasing (above a short term MA) +1
 RS zero line rising +1
 RS zero line falling -1
 RS falling (below a short term MA) -1
 RS below zero line -2

The breadth plot would be the average score.  Trouble with this plan, I'd have to run the breadth calcs again...

Swapping the index is easy.  I'll change the index to the S&P Small cap 600 later.  Don't seem to have the Russell 2000 available, except for a UK ETF which may have currency exchange rates overlaid - confusing the picture.

Not what you suggested, but swapping the underlying breadth pool is slower.  It was about 2 1/2 mins per point for these latest calcs.  My computer started to swap memory much more, so my plan to see if I can load all of the stocks in breadth pool to allow for MUCH faster historical breath calculations is probably a non-starter.  I can try loading the data to see if it is feasible fairly quickly though.

I'm only referring to visual referencing not changing the calculation.

I've got a plot of the S&P Small Cap 600 against my stage and Mansfield RS plots below.  That was a good call.  Significant point, the % stocks with their RS above the zero line (orange) hits its peak just as the S&P600 stops rising.  So the while the S&P600 is on a plateau the orange line is descending as are the number of Stage 2 stocks, Stage 3 starts to grow.  The S&P600 goes into Stage 4 a bit after after the % stocks with a RS zero line rising (lighter blue) crosses down through the 50% level.
   

On a separate note, with the development of the screener on the front end of the website I will have many years of Mansfield RS data for every US stock soon, and other markets potentially. UK, India etc. But starting with the US data. So i'll be able to do some backtests on the data.

I presume you have your own database then?  At some point I found it more flexible for individual stocks to calculate this sort of data on the fly.  But, then calculating the market breadth is slower as you have to keep recalculating the data for every stock in the breadth pool.  The problem I had hit with carrying out all the calculations and then storing them was that (I was using python) that the  calculations of the various parameters, per stock, were taking too long to calculate and store in the database.  But perhaps serious speedups were possible?

[pcabc]

There is some correlation of this discussion with a plot of the relative strenght (not Mansfield) of the S&P600 (small cap) against the S&P500 (large cap):
   

THis is good as it implies that the weight of evidence approach it likely to work when what you see in one breadth indicator matches what is seen in the indicies.

(not that I ever doubted it, but confirmation that things correlate as expected builds confidence).

[isatrader]

I presume you have your own database then?  At some point I found it more flexible for individual stocks to calculate this sort of data on the fly.  But, then calculating the market breadth is slower as you have to keep recalculating the data for every stock in the breadth pool.  The problem I had hit with carrying out all the calculations and then storing them was that (I was using python) that the  calculations of the various parameters, per stock, were taking too long to calculate and store in the database.  But perhaps serious speedups were possible?

Yep, we have a cloud server setup that will be pulling in the data each day into the screener. But just trying to sort out the data provider, as for commercial use the data can be very expensive each month. So trying to find a provider that is reasonable that will provide the adjusted daily and weekly data, plus sector and group info.

Here's an example of what the charts will look like. Each stock will have it's own page.

       

[pcabc]

Yep, we have a cloud server setup that will be pulling in the data each day into the screener. But just trying to sort out the data provider, as for commercial use the data can be very expensive each month. So trying to find a provider that is reasonable that will provide the adjusted daily and weekly data, plus sector and group info.

Here's an example of what the charts will look like. Each stock will have it's own page.

I found that pulling data is not as simple as it sounds.  Its easy until you find that there are market holidays which vary by market, that cryptos and forex (?) have data on weekends where stocks and commodities do not.  In the end I decided to just ignore market holidays, it was simpler.  So for market breadth purposes I make a copy of the indices and fill in the gaps caused by market holidays.  Another piece of fun is that sometimes stock data can be slow, even perhaps for individual stocks.  So I bulk download end of day data.  Then at some point overnight I have a script that checks for stocks missing data and it tries to update that data individually.  If a stock fails to be updated that way 10 times in a row then I assume that stock ticker is dead.

The smaller cap stocks seem to have more data issues than the larger ones.  Perhaps they simply don't trade on some days?

When free data was available Yahoo had a reasonable API, Google was second.  When they stopped I found it took me a while to find a provider at an affordable price.  Unsure whether some were commercial feeds rather than for individuals, but the costs could escalate quickly.

As for sector and group info.  There is some free data for some markets that I can find from time to time.  But each source has its own format. I could get the data from my existing data provider, but I would have to update my package to one four times the price, so I'm not keen to go that route right now.


Nice clear charts.

[isatrader]

I found that pulling data is not as simple as it sounds.  Its easy until you find that there are market holidays which vary by market, that cryptos and forex (?) have data on weekends where stocks and commodities do not.  In the end I decided to just ignore market holidays, it was simpler.  So for market breadth purposes I make a copy of the indices and fill in the gaps caused by market holidays.  Another piece of fun is that sometimes stock data can be slow, even perhaps for individual stocks.  So I bulk download end of day data.  Then at some point overnight I have a script that checks for stocks missing data and it tries to update that data individually.  If a stock fails to be updated that way 10 times in a row then I assume that stock ticker is dead.

The smaller cap stocks seem to have more data issues than the larger ones.  Perhaps they simply don't trade on some days?

When free data was available Yahoo had a reasonable API, Google was second.  When they stopped I found it took me a while to find a provider at an affordable price.  Unsure whether some were commercial feeds rather than for individuals, but the costs could escalate quickly.

As for sector and group info.  There is some free data for some markets that I can find from time to time.  But each source has its own format. I could get the data from my existing data provider, but I would have to update my package to one four times the price, so I'm not keen to go that route right now.

Nice clear charts.

My developer has been working on it since Christmas, so is a big endeavour, but is initially only going to be focused on the US markets, as data costs for a commercial licence are at least ten times that of those for personal use per month. So am currently negotiating with a number of data providers to try and get what I need and to make the product viable for me. As the first quote I got came in at $1250 a month, which is just for open, high, low, close, volume data. So was concerned that I wouldn't be able to get it done, but have since found a couple of providers willing to do it in the hundreds, not thousands a month. As redistributing data on a website is very different price wise than personal use. But hopefully, with the higher price, also comes more sanitised data for errors, like you've mentioned. But we'll see in time with that.

[pcabc]

My developer has been working on it since Christmas, so is a big endeavour, but is initially only going to be focused on the US markets, as data costs for a commercial licence are at least ten times that of those for personal use per month. So am currently negotiating with a number of data providers to try and get what I need and to make the product viable for me. As the first quote I got came in at $1250 a month, which is just for open, high, low, close, volume data. So was concerned that I wouldn't be able to get it done, but have since found a couple of providers willing to do it in the hundreds, not thousands a month. As redistributing data on a website is very different price wise than personal use. But hopefully, with the higher price, also comes more sanitised data for errors, like you've mentioned. But we'll see in time with that.

That is a massive leap there, I really hope it works well.  I suppose the problem is that things are either tailored for individual traders or institutions, and even the smallest institution will have deep pockets.  But that is supposition?  Starting with the US market has got to make sense, it will get the most attention in the English and likely non-English speaking worlds.  The performance in the UK is nowhere near it no matter how nice it would be to include it, and even if UK performance had matched or exceeded then for the aforementioned reasons the US would need to be the focus.  Plus, at least my screening experience is that very good Stage Analysis screens can be rare beasts - you really do need to trawl a really wide pool and the US pool of stocks is the largest.

I'm unsurprised by the amount of effort.  Over the years I've put many many man hours in my database / screener.  Yes, my screener 'happened' rather than was planned.  But if I had to start from fresh tomorrow it would take a massive chunk of time.  Plus - what I do has less pressure to look nice and I can tolerate quirks as its not public or customer facing.

Frustratingly I've implemented additional Mansfield RS breadth measures that I mentioned in earlier posts - but this morning - coincident with a minor tweak that I soon reversed, it stopped working.  No new breadth calcs - at least the ones I wanted, appeared in my database.  I think I'm low on ram, likely am for what I am doing.  Low and behold, a couple of hours later it seems to be working - nothing to do with any fixes I tried.  But only after having ordered more RAM.  So either I've splashed out on quadrupling my RAM for only a marginal benefit and I will kick myself for a marginal purchase - or it will give me a massive speedup and I will kick myself for not doing it sooner.  There is a hint that it was the web browser making the very slow to process request might be the issue?

(if anyone suggests I'm a fool for spending more money on RAM when it was working with the same amount of RAM hours earlier, I note that machines can really bog down and become unusable if they run low on RAM and swap a lot.  I think my machine may have been marginal and just tipped over the edge.  Furthermore, there is not much I could do about memory usage without a large investment of time and inconvenience, cutting capability and then I'd probably need more RAM anyway.)

It will be interesting to see how what you are doing scales? Hopefully cloud will help?  And your developer is extremely likely to know more about this than my amateur efforts.  On my personal machine here, as its operating both as the client machine, web server and database server (as this seemed a simple way to do things) I think that provided its not doing the Mansfield RS calculations it could likely serve more than a single user if I were so inclined and data licenses were no issue.  But it definitely starts to bog down with the breadth calcs.  Furthermore, it was hard to see if it was memory, CPU or IO limited.  It seemed to be frustratingly, relying on few threads, to although I've an 8-core 16 thread machine more cores did not necessarily help with speed.

So, hopefully your endeavors go well, and I've fixed my issues and increasing the RAM gives me a good speedup.

[pcabc]

Breadth based on Mansfield Relative Strength - Relative Strength Rising or Falling

I attach a new calculation, the percentage of stocks with the Mansfield Relative Strength rising or falling compared to the index.  This is plot [4] in the attached chart.  The definition of rising or falling is whether the 50 day simple moving average of the Relative Strength is rising or falling.  I tried it first with a 5 day MA, which I had already calculated, but that fluctuated much faster then the Percentage of Stocks with their Mansfield RS above zero and % Stocks with the Mansfield Zero Line rising.

   

I've only calculated this for a year, I need to extend the period I do not think a one year sample is sufficient to see if this indicator is of value.  I think at the moment the best I can tell is that it is best read in conjunction with other to RS based indicators.  I could perhaps try variants on 20, 50 and 150 day periods to compare them, at probably a relatively small time and complexity disadvantage.  The question is, where do I stop?


Looking at this specific chart I'm not sure the RS rising tells us much when the S&P500 is in Stage 4.  However, from November 2022 it is positive alongside the % Mansfield RS above Zero [2] and Mansfield RS Zero Line Rising [3].  It has an even more dramatic drop below 50% in March 2023.  Along with the other indicators this implies that only the large caps are holding the index up.