How to handle many time series?Meaning of Moving Average Term in ARIMA
Understanding a part of the proof that sequence that converges to square root of two is decreasing.
Good way to stop electrolyte tabs from turning into powder?
Why does Japan use the same type of AC power outlet as the US?
Insert or push_back to end of a std::vector?
How do ultra-stable oscillators for spacecraft work?
May the tower use the runway while an emergency aircraft is inbound?
What is the opposite of "hunger level"?
What modifiers are added to the attack and damage rolls of this unique longbow from Waterdeep: Dragon Heist?
What is the spellcasting ability of a Barbarian Totem Warrior?
What was the intention with the Commodore 128?
Is the Microsoft recommendation to use C# properties applicable to game development?
What is the question mark?
Can anyone help me what's wrong here as i can prove 0 = 1?
How to get locks that are keyed alike?
Ghost house where the house only appeared once a year for it was the ghost
Visa on arrival to exit airport in Russia
Does Reckless Attack work with Multiattack when wild shaped?
How to gracefully leave a company you helped start?
Physical Interpretation of an Overdamped Pendulum
What are these panels underneath the wing root of a A380?
Is there any official ruling on how characters go from 0th to 1st level in a class?
Is this bar slide trick shown on Cheers real or a visual effect?
What are some tips and tricks for finding the cheapest flight when luggage and other fees are not revealed until far into the booking process?
Doesn't the speed of light limit imply the same electron can be annihilated twice?
How to handle many time series?
Meaning of Moving Average Term in ARIMA
$begingroup$
At first, I should apologize if this question is not relevant to this website, but since there are some researchers from the management science community, I ask the question here.
I have data for the demand of 1200 products for 25 periods. That is, 1200 time series. I want to predict the demand for each product for the next period (26). However, because of the high-complexity of tuning the parameters $(p,d,q)$ of the ARIMA model, it is not possible to use the ARIMA model.
I have used the time-slicing approach to train ML approaches (random forest, xgboost, catboost, ....) to predict the demand, but it is satisfactory. I would like to know if there is any other approach for demand prediction of 1200 products?
Edit: I have used the following approach. I would be thankful if someone can suggest any idea. At first, I have generated new feature (trend) for each time-slicing time serie as if the demand of a product for a period is increase compared to the previous period (+1 for increase, 0 no difference, -1 decrease) and then clustered time-slicing time series based on the trend data (not the value of demand) and then predict the demand of each product according to the demand of products which fall into the same cluster with KNN. KNN algorithm is trained on the demand values but clustering is trained on trend data. KNN algorithm with a large number of neighbors produces good results but other algorithms such as catboost, xgboost, knn with small k produces poor results.
time-series forecasting
$endgroup$
|
show 1 more comment
$begingroup$
At first, I should apologize if this question is not relevant to this website, but since there are some researchers from the management science community, I ask the question here.
I have data for the demand of 1200 products for 25 periods. That is, 1200 time series. I want to predict the demand for each product for the next period (26). However, because of the high-complexity of tuning the parameters $(p,d,q)$ of the ARIMA model, it is not possible to use the ARIMA model.
I have used the time-slicing approach to train ML approaches (random forest, xgboost, catboost, ....) to predict the demand, but it is satisfactory. I would like to know if there is any other approach for demand prediction of 1200 products?
Edit: I have used the following approach. I would be thankful if someone can suggest any idea. At first, I have generated new feature (trend) for each time-slicing time serie as if the demand of a product for a period is increase compared to the previous period (+1 for increase, 0 no difference, -1 decrease) and then clustered time-slicing time series based on the trend data (not the value of demand) and then predict the demand of each product according to the demand of products which fall into the same cluster with KNN. KNN algorithm is trained on the demand values but clustering is trained on trend data. KNN algorithm with a large number of neighbors produces good results but other algorithms such as catboost, xgboost, knn with small k produces poor results.
time-series forecasting
$endgroup$
7
$begingroup$
Perhaps Stats.SE has more information on time series.
$endgroup$
– TheSimpliFire♦
Aug 4 at 10:39
$begingroup$
Can't you just train 1200 independent models, one for each product?
$endgroup$
– Simon
Aug 4 at 11:45
$begingroup$
@Simon Since it is needed to tune (p,d,q) parameters, it takes a long time and it is neither acceptable nor applicable.
$endgroup$
– Amin Sh
Aug 4 at 12:09
2
$begingroup$
Here is a list of many sych posts at CrossValidated.
$endgroup$
– kjetil b halvorsen
Aug 4 at 19:12
3
$begingroup$
@kjetil b halvorsen You seem to be carrying over the practice from Cross Validated of editing posts to remove "Thanks". I think it's stupid to "disallow" "Thanks" and even stupider for someone to edit someone else's post just to remove it. Nevertheless, that seems to be the rule at Cross Validated, so I accepted it there. it doesn't seem to be the rule on this forum, and 'd rather not see it become the rule or practice here. I'm generally against editing other people's posts unless there's a good reason, and I don't think removing "Thanks" is a good reason. Peace out.
$endgroup$
– Mark L. Stone
Aug 5 at 1:26
|
show 1 more comment
$begingroup$
At first, I should apologize if this question is not relevant to this website, but since there are some researchers from the management science community, I ask the question here.
I have data for the demand of 1200 products for 25 periods. That is, 1200 time series. I want to predict the demand for each product for the next period (26). However, because of the high-complexity of tuning the parameters $(p,d,q)$ of the ARIMA model, it is not possible to use the ARIMA model.
I have used the time-slicing approach to train ML approaches (random forest, xgboost, catboost, ....) to predict the demand, but it is satisfactory. I would like to know if there is any other approach for demand prediction of 1200 products?
Edit: I have used the following approach. I would be thankful if someone can suggest any idea. At first, I have generated new feature (trend) for each time-slicing time serie as if the demand of a product for a period is increase compared to the previous period (+1 for increase, 0 no difference, -1 decrease) and then clustered time-slicing time series based on the trend data (not the value of demand) and then predict the demand of each product according to the demand of products which fall into the same cluster with KNN. KNN algorithm is trained on the demand values but clustering is trained on trend data. KNN algorithm with a large number of neighbors produces good results but other algorithms such as catboost, xgboost, knn with small k produces poor results.
time-series forecasting
$endgroup$
At first, I should apologize if this question is not relevant to this website, but since there are some researchers from the management science community, I ask the question here.
I have data for the demand of 1200 products for 25 periods. That is, 1200 time series. I want to predict the demand for each product for the next period (26). However, because of the high-complexity of tuning the parameters $(p,d,q)$ of the ARIMA model, it is not possible to use the ARIMA model.
I have used the time-slicing approach to train ML approaches (random forest, xgboost, catboost, ....) to predict the demand, but it is satisfactory. I would like to know if there is any other approach for demand prediction of 1200 products?
Edit: I have used the following approach. I would be thankful if someone can suggest any idea. At first, I have generated new feature (trend) for each time-slicing time serie as if the demand of a product for a period is increase compared to the previous period (+1 for increase, 0 no difference, -1 decrease) and then clustered time-slicing time series based on the trend data (not the value of demand) and then predict the demand of each product according to the demand of products which fall into the same cluster with KNN. KNN algorithm is trained on the demand values but clustering is trained on trend data. KNN algorithm with a large number of neighbors produces good results but other algorithms such as catboost, xgboost, knn with small k produces poor results.
time-series forecasting
time-series forecasting
edited Aug 5 at 14:01
Amin Sh
asked Aug 4 at 10:35
Amin ShAmin Sh
3931 silver badge9 bronze badges
3931 silver badge9 bronze badges
7
$begingroup$
Perhaps Stats.SE has more information on time series.
$endgroup$
– TheSimpliFire♦
Aug 4 at 10:39
$begingroup$
Can't you just train 1200 independent models, one for each product?
$endgroup$
– Simon
Aug 4 at 11:45
$begingroup$
@Simon Since it is needed to tune (p,d,q) parameters, it takes a long time and it is neither acceptable nor applicable.
$endgroup$
– Amin Sh
Aug 4 at 12:09
2
$begingroup$
Here is a list of many sych posts at CrossValidated.
$endgroup$
– kjetil b halvorsen
Aug 4 at 19:12
3
$begingroup$
@kjetil b halvorsen You seem to be carrying over the practice from Cross Validated of editing posts to remove "Thanks". I think it's stupid to "disallow" "Thanks" and even stupider for someone to edit someone else's post just to remove it. Nevertheless, that seems to be the rule at Cross Validated, so I accepted it there. it doesn't seem to be the rule on this forum, and 'd rather not see it become the rule or practice here. I'm generally against editing other people's posts unless there's a good reason, and I don't think removing "Thanks" is a good reason. Peace out.
$endgroup$
– Mark L. Stone
Aug 5 at 1:26
|
show 1 more comment
7
$begingroup$
Perhaps Stats.SE has more information on time series.
$endgroup$
– TheSimpliFire♦
Aug 4 at 10:39
$begingroup$
Can't you just train 1200 independent models, one for each product?
$endgroup$
– Simon
Aug 4 at 11:45
$begingroup$
@Simon Since it is needed to tune (p,d,q) parameters, it takes a long time and it is neither acceptable nor applicable.
$endgroup$
– Amin Sh
Aug 4 at 12:09
2
$begingroup$
Here is a list of many sych posts at CrossValidated.
$endgroup$
– kjetil b halvorsen
Aug 4 at 19:12
3
$begingroup$
@kjetil b halvorsen You seem to be carrying over the practice from Cross Validated of editing posts to remove "Thanks". I think it's stupid to "disallow" "Thanks" and even stupider for someone to edit someone else's post just to remove it. Nevertheless, that seems to be the rule at Cross Validated, so I accepted it there. it doesn't seem to be the rule on this forum, and 'd rather not see it become the rule or practice here. I'm generally against editing other people's posts unless there's a good reason, and I don't think removing "Thanks" is a good reason. Peace out.
$endgroup$
– Mark L. Stone
Aug 5 at 1:26
7
7
$begingroup$
Perhaps Stats.SE has more information on time series.
$endgroup$
– TheSimpliFire♦
Aug 4 at 10:39
$begingroup$
Perhaps Stats.SE has more information on time series.
$endgroup$
– TheSimpliFire♦
Aug 4 at 10:39
$begingroup$
Can't you just train 1200 independent models, one for each product?
$endgroup$
– Simon
Aug 4 at 11:45
$begingroup$
Can't you just train 1200 independent models, one for each product?
$endgroup$
– Simon
Aug 4 at 11:45
$begingroup$
@Simon Since it is needed to tune (p,d,q) parameters, it takes a long time and it is neither acceptable nor applicable.
$endgroup$
– Amin Sh
Aug 4 at 12:09
$begingroup$
@Simon Since it is needed to tune (p,d,q) parameters, it takes a long time and it is neither acceptable nor applicable.
$endgroup$
– Amin Sh
Aug 4 at 12:09
2
2
$begingroup$
Here is a list of many sych posts at CrossValidated.
$endgroup$
– kjetil b halvorsen
Aug 4 at 19:12
$begingroup$
Here is a list of many sych posts at CrossValidated.
$endgroup$
– kjetil b halvorsen
Aug 4 at 19:12
3
3
$begingroup$
@kjetil b halvorsen You seem to be carrying over the practice from Cross Validated of editing posts to remove "Thanks". I think it's stupid to "disallow" "Thanks" and even stupider for someone to edit someone else's post just to remove it. Nevertheless, that seems to be the rule at Cross Validated, so I accepted it there. it doesn't seem to be the rule on this forum, and 'd rather not see it become the rule or practice here. I'm generally against editing other people's posts unless there's a good reason, and I don't think removing "Thanks" is a good reason. Peace out.
$endgroup$
– Mark L. Stone
Aug 5 at 1:26
$begingroup$
@kjetil b halvorsen You seem to be carrying over the practice from Cross Validated of editing posts to remove "Thanks". I think it's stupid to "disallow" "Thanks" and even stupider for someone to edit someone else's post just to remove it. Nevertheless, that seems to be the rule at Cross Validated, so I accepted it there. it doesn't seem to be the rule on this forum, and 'd rather not see it become the rule or practice here. I'm generally against editing other people's posts unless there's a good reason, and I don't think removing "Thanks" is a good reason. Peace out.
$endgroup$
– Mark L. Stone
Aug 5 at 1:26
|
show 1 more comment
3 Answers
3
active
oldest
votes
$begingroup$
If the 1200 products are closely related, so that trend (if any) and noise are likely to be correlated across products, a single model might make sense. If they are loosely related (so that they might share a common trend but separate noise processes), you might consider fitting a single trend model (linear regression on time?), "detrending" the data, then fitting separate ARMA models ... or perhaps just separate exponential smoothing models? If the products are unrelated, either separate ARIMA models or separate exponential smoothing models would be warranted, and a composite model would not be.
$endgroup$
add a comment |
$begingroup$
This problem is a multivariate (simply when you have more than one time-dependent variables) time series for which you can use Vector Auto Regression (VAR) technique among some others. Explanation and Python implementation of this technique has been discussed in details in here. This technique is also considered the dependencies between various time-dependent variables in the $AR(n)$ calculations.
I am not pretty sure that the implementation code will take less than your own approaches for the forecasting of variables but at least it gives you some hints to accelerate other approaches. (of course, it also required some familiarities with univariate time-series and Python programming language).
$endgroup$
add a comment |
$begingroup$
The demand of 1200 product will often be related. There might be common events (Christmas, some large accident, ...) that influence all or many of the demands, substitution effects, ... or there may be relations imposed by the cost function (inventory control ...)
These can be tackled by some common model, and this is often hierarchical forecasting. There are some posts on Cross Validated, and rather than rewrite here, this is a list.
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "700"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2for.stackexchange.com%2fquestions%2f1154%2fhow-to-handle-many-time-series%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If the 1200 products are closely related, so that trend (if any) and noise are likely to be correlated across products, a single model might make sense. If they are loosely related (so that they might share a common trend but separate noise processes), you might consider fitting a single trend model (linear regression on time?), "detrending" the data, then fitting separate ARMA models ... or perhaps just separate exponential smoothing models? If the products are unrelated, either separate ARIMA models or separate exponential smoothing models would be warranted, and a composite model would not be.
$endgroup$
add a comment |
$begingroup$
If the 1200 products are closely related, so that trend (if any) and noise are likely to be correlated across products, a single model might make sense. If they are loosely related (so that they might share a common trend but separate noise processes), you might consider fitting a single trend model (linear regression on time?), "detrending" the data, then fitting separate ARMA models ... or perhaps just separate exponential smoothing models? If the products are unrelated, either separate ARIMA models or separate exponential smoothing models would be warranted, and a composite model would not be.
$endgroup$
add a comment |
$begingroup$
If the 1200 products are closely related, so that trend (if any) and noise are likely to be correlated across products, a single model might make sense. If they are loosely related (so that they might share a common trend but separate noise processes), you might consider fitting a single trend model (linear regression on time?), "detrending" the data, then fitting separate ARMA models ... or perhaps just separate exponential smoothing models? If the products are unrelated, either separate ARIMA models or separate exponential smoothing models would be warranted, and a composite model would not be.
$endgroup$
If the 1200 products are closely related, so that trend (if any) and noise are likely to be correlated across products, a single model might make sense. If they are loosely related (so that they might share a common trend but separate noise processes), you might consider fitting a single trend model (linear regression on time?), "detrending" the data, then fitting separate ARMA models ... or perhaps just separate exponential smoothing models? If the products are unrelated, either separate ARIMA models or separate exponential smoothing models would be warranted, and a composite model would not be.
answered Aug 4 at 17:00
prubinprubin
3,1515 silver badges24 bronze badges
3,1515 silver badges24 bronze badges
add a comment |
add a comment |
$begingroup$
This problem is a multivariate (simply when you have more than one time-dependent variables) time series for which you can use Vector Auto Regression (VAR) technique among some others. Explanation and Python implementation of this technique has been discussed in details in here. This technique is also considered the dependencies between various time-dependent variables in the $AR(n)$ calculations.
I am not pretty sure that the implementation code will take less than your own approaches for the forecasting of variables but at least it gives you some hints to accelerate other approaches. (of course, it also required some familiarities with univariate time-series and Python programming language).
$endgroup$
add a comment |
$begingroup$
This problem is a multivariate (simply when you have more than one time-dependent variables) time series for which you can use Vector Auto Regression (VAR) technique among some others. Explanation and Python implementation of this technique has been discussed in details in here. This technique is also considered the dependencies between various time-dependent variables in the $AR(n)$ calculations.
I am not pretty sure that the implementation code will take less than your own approaches for the forecasting of variables but at least it gives you some hints to accelerate other approaches. (of course, it also required some familiarities with univariate time-series and Python programming language).
$endgroup$
add a comment |
$begingroup$
This problem is a multivariate (simply when you have more than one time-dependent variables) time series for which you can use Vector Auto Regression (VAR) technique among some others. Explanation and Python implementation of this technique has been discussed in details in here. This technique is also considered the dependencies between various time-dependent variables in the $AR(n)$ calculations.
I am not pretty sure that the implementation code will take less than your own approaches for the forecasting of variables but at least it gives you some hints to accelerate other approaches. (of course, it also required some familiarities with univariate time-series and Python programming language).
$endgroup$
This problem is a multivariate (simply when you have more than one time-dependent variables) time series for which you can use Vector Auto Regression (VAR) technique among some others. Explanation and Python implementation of this technique has been discussed in details in here. This technique is also considered the dependencies between various time-dependent variables in the $AR(n)$ calculations.
I am not pretty sure that the implementation code will take less than your own approaches for the forecasting of variables but at least it gives you some hints to accelerate other approaches. (of course, it also required some familiarities with univariate time-series and Python programming language).
answered Aug 4 at 14:58
Oguz ToragayOguz Toragay
2,2472 silver badges26 bronze badges
2,2472 silver badges26 bronze badges
add a comment |
add a comment |
$begingroup$
The demand of 1200 product will often be related. There might be common events (Christmas, some large accident, ...) that influence all or many of the demands, substitution effects, ... or there may be relations imposed by the cost function (inventory control ...)
These can be tackled by some common model, and this is often hierarchical forecasting. There are some posts on Cross Validated, and rather than rewrite here, this is a list.
$endgroup$
add a comment |
$begingroup$
The demand of 1200 product will often be related. There might be common events (Christmas, some large accident, ...) that influence all or many of the demands, substitution effects, ... or there may be relations imposed by the cost function (inventory control ...)
These can be tackled by some common model, and this is often hierarchical forecasting. There are some posts on Cross Validated, and rather than rewrite here, this is a list.
$endgroup$
add a comment |
$begingroup$
The demand of 1200 product will often be related. There might be common events (Christmas, some large accident, ...) that influence all or many of the demands, substitution effects, ... or there may be relations imposed by the cost function (inventory control ...)
These can be tackled by some common model, and this is often hierarchical forecasting. There are some posts on Cross Validated, and rather than rewrite here, this is a list.
$endgroup$
The demand of 1200 product will often be related. There might be common events (Christmas, some large accident, ...) that influence all or many of the demands, substitution effects, ... or there may be relations imposed by the cost function (inventory control ...)
These can be tackled by some common model, and this is often hierarchical forecasting. There are some posts on Cross Validated, and rather than rewrite here, this is a list.
answered Aug 4 at 19:40
kjetil b halvorsenkjetil b halvorsen
3753 silver badges12 bronze badges
3753 silver badges12 bronze badges
add a comment |
add a comment |
Thanks for contributing an answer to Operations Research Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2for.stackexchange.com%2fquestions%2f1154%2fhow-to-handle-many-time-series%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
7
$begingroup$
Perhaps Stats.SE has more information on time series.
$endgroup$
– TheSimpliFire♦
Aug 4 at 10:39
$begingroup$
Can't you just train 1200 independent models, one for each product?
$endgroup$
– Simon
Aug 4 at 11:45
$begingroup$
@Simon Since it is needed to tune (p,d,q) parameters, it takes a long time and it is neither acceptable nor applicable.
$endgroup$
– Amin Sh
Aug 4 at 12:09
2
$begingroup$
Here is a list of many sych posts at CrossValidated.
$endgroup$
– kjetil b halvorsen
Aug 4 at 19:12
3
$begingroup$
@kjetil b halvorsen You seem to be carrying over the practice from Cross Validated of editing posts to remove "Thanks". I think it's stupid to "disallow" "Thanks" and even stupider for someone to edit someone else's post just to remove it. Nevertheless, that seems to be the rule at Cross Validated, so I accepted it there. it doesn't seem to be the rule on this forum, and 'd rather not see it become the rule or practice here. I'm generally against editing other people's posts unless there's a good reason, and I don't think removing "Thanks" is a good reason. Peace out.
$endgroup$
– Mark L. Stone
Aug 5 at 1:26