How to keep products synchronized in two shopsExporting admin panel settings that are saved to the databaseIssue when Exporting Products CatalogHow to modify the export function to keep formulaHow to import Etsy products into Magento?Reduce DB file sizeHow to Move Products, Attributes, Images, and Customers From One Magento Installation to AnotherHow to compare products from two categoriesImported Products Not In Catalog --> Manage ProductsHow to Import new categories and products from one site to another without overriding the existingBest way to bulk update stock and price for 50+ thousands of products?
Employer wants to use my work email account after I quit
Suggested order for Amazon Prime Doctor Who series
Methodology: Writing unit tests for another developer
How would modern naval warfare have to have developed differently for battleships to still be relevant in the 21st century?
Is there a term for the belief that "if it's legal, it's moral"?
What's currently blocking the construction of the wall between Mexico and the US?
Trainee keeps missing deadlines for independent learning
Drawing people along with x and y axis
Why do all the teams that I have worked with always finish a sprint without completion of all the stories?
What does "play with your toy’s toys" mean?
What is the meaning of "понаехать"?
Is this proposal by U.S. presidential candidate Pete Buttigieg to change the composition of the Supreme Court constitutional?
What does the hyphen "-" mean in "tar xzf -"?
Can someone suggest a path to study Mordell-Weil theorem for someone studying on their own?
Why don't countries like Japan just print more money?
What's the difference between a deep fryer and a chip pan?
Find the C-factor of a vote
What is the origin of Scooby-Doo's name?
What exactly is the 'online' in OLAP and OLTP?
Impossible darts scores
Can humans ever directly see a few photons at a time? Can a human see a single photon?
Who are the remaining King/Queenslayers?
.NET executes a SQL query and Active Monitor shows multiple rows blocking each other
Dates on degrees don’t make sense – will people care?
How to keep products synchronized in two shops
Exporting admin panel settings that are saved to the databaseIssue when Exporting Products CatalogHow to modify the export function to keep formulaHow to import Etsy products into Magento?Reduce DB file sizeHow to Move Products, Attributes, Images, and Customers From One Magento Installation to AnotherHow to compare products from two categoriesImported Products Not In Catalog --> Manage ProductsHow to Import new categories and products from one site to another without overriding the existingBest way to bulk update stock and price for 50+ thousands of products?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
How can I transfer all products from one shop to another after clicking a button?
We have a shop which is a copy of our real shop. We need to keep the products, prices, etc. synchronized in both shops.,
The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live
It would be best if we could just press a button which starts the import process.
magento-1.9 database import catalog export
This question has an open bounty worth +50
reputation from Black ending ending at 2019-06-25 07:13:21Z">in 5 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
|
show 5 more comments
How can I transfer all products from one shop to another after clicking a button?
We have a shop which is a copy of our real shop. We need to keep the products, prices, etc. synchronized in both shops.,
The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live
It would be best if we could just press a button which starts the import process.
magento-1.9 database import catalog export
This question has an open bounty worth +50
reputation from Black ending ending at 2019-06-25 07:13:21Z">in 5 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
sounds like a use-case for using the multi-store capability of magento rather than having two separate instances that need to be synced.
– BlueC
Jun 14 at 12:37
@BlueC, thats not possible since both shops are using different code (different git branches checked out)
– Black
2 days ago
1
What's different about the code? Can you unify the changes using themes/child themes etc? Magento is built to do this.
– BlueC
2 days ago
1
Hmmm, templates and scripts can all be utilised per store quite easily. Extensions can be a pain because they can conflict. Observers could/should be written to be configurable per store view. I am not saying its impossible to synchronize the products, but it doesn't seem a very elegant solution, and longer term having a single store with unified products might be preferable. Does sound like it might be tricky to untangle though.
– BlueC
2 days ago
1
Reading below it seems you don't actually want or need 2 storefronts, but you just want a staging and production site and to be able to sync from production to staging. This wasn't clear in your question, and in this case using the multi-store setup in Magento is not appropriate.
– BlueC
19 hours ago
|
show 5 more comments
How can I transfer all products from one shop to another after clicking a button?
We have a shop which is a copy of our real shop. We need to keep the products, prices, etc. synchronized in both shops.,
The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live
It would be best if we could just press a button which starts the import process.
magento-1.9 database import catalog export
How can I transfer all products from one shop to another after clicking a button?
We have a shop which is a copy of our real shop. We need to keep the products, prices, etc. synchronized in both shops.,
The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live
It would be best if we could just press a button which starts the import process.
magento-1.9 database import catalog export
magento-1.9 database import catalog export
edited 18 hours ago
Black
asked Jun 13 at 12:45
BlackBlack
432420
432420
This question has an open bounty worth +50
reputation from Black ending ending at 2019-06-25 07:13:21Z">in 5 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
This question has an open bounty worth +50
reputation from Black ending ending at 2019-06-25 07:13:21Z">in 5 days.
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
sounds like a use-case for using the multi-store capability of magento rather than having two separate instances that need to be synced.
– BlueC
Jun 14 at 12:37
@BlueC, thats not possible since both shops are using different code (different git branches checked out)
– Black
2 days ago
1
What's different about the code? Can you unify the changes using themes/child themes etc? Magento is built to do this.
– BlueC
2 days ago
1
Hmmm, templates and scripts can all be utilised per store quite easily. Extensions can be a pain because they can conflict. Observers could/should be written to be configurable per store view. I am not saying its impossible to synchronize the products, but it doesn't seem a very elegant solution, and longer term having a single store with unified products might be preferable. Does sound like it might be tricky to untangle though.
– BlueC
2 days ago
1
Reading below it seems you don't actually want or need 2 storefronts, but you just want a staging and production site and to be able to sync from production to staging. This wasn't clear in your question, and in this case using the multi-store setup in Magento is not appropriate.
– BlueC
19 hours ago
|
show 5 more comments
sounds like a use-case for using the multi-store capability of magento rather than having two separate instances that need to be synced.
– BlueC
Jun 14 at 12:37
@BlueC, thats not possible since both shops are using different code (different git branches checked out)
– Black
2 days ago
1
What's different about the code? Can you unify the changes using themes/child themes etc? Magento is built to do this.
– BlueC
2 days ago
1
Hmmm, templates and scripts can all be utilised per store quite easily. Extensions can be a pain because they can conflict. Observers could/should be written to be configurable per store view. I am not saying its impossible to synchronize the products, but it doesn't seem a very elegant solution, and longer term having a single store with unified products might be preferable. Does sound like it might be tricky to untangle though.
– BlueC
2 days ago
1
Reading below it seems you don't actually want or need 2 storefronts, but you just want a staging and production site and to be able to sync from production to staging. This wasn't clear in your question, and in this case using the multi-store setup in Magento is not appropriate.
– BlueC
19 hours ago
sounds like a use-case for using the multi-store capability of magento rather than having two separate instances that need to be synced.
– BlueC
Jun 14 at 12:37
sounds like a use-case for using the multi-store capability of magento rather than having two separate instances that need to be synced.
– BlueC
Jun 14 at 12:37
@BlueC, thats not possible since both shops are using different code (different git branches checked out)
– Black
2 days ago
@BlueC, thats not possible since both shops are using different code (different git branches checked out)
– Black
2 days ago
1
1
What's different about the code? Can you unify the changes using themes/child themes etc? Magento is built to do this.
– BlueC
2 days ago
What's different about the code? Can you unify the changes using themes/child themes etc? Magento is built to do this.
– BlueC
2 days ago
1
1
Hmmm, templates and scripts can all be utilised per store quite easily. Extensions can be a pain because they can conflict. Observers could/should be written to be configurable per store view. I am not saying its impossible to synchronize the products, but it doesn't seem a very elegant solution, and longer term having a single store with unified products might be preferable. Does sound like it might be tricky to untangle though.
– BlueC
2 days ago
Hmmm, templates and scripts can all be utilised per store quite easily. Extensions can be a pain because they can conflict. Observers could/should be written to be configurable per store view. I am not saying its impossible to synchronize the products, but it doesn't seem a very elegant solution, and longer term having a single store with unified products might be preferable. Does sound like it might be tricky to untangle though.
– BlueC
2 days ago
1
1
Reading below it seems you don't actually want or need 2 storefronts, but you just want a staging and production site and to be able to sync from production to staging. This wasn't clear in your question, and in this case using the multi-store setup in Magento is not appropriate.
– BlueC
19 hours ago
Reading below it seems you don't actually want or need 2 storefronts, but you just want a staging and production site and to be able to sync from production to staging. This wasn't clear in your question, and in this case using the multi-store setup in Magento is not appropriate.
– BlueC
19 hours ago
|
show 5 more comments
3 Answers
3
active
oldest
votes
not sure how your copy store works, but you can use many options:
Magento Sync Products paid extension- Odoo connector Example free connector/ Odoo bridge Example Webkul paid extension
Magmi Magento Mass Importer- Store manager for magento
- Magento Dataflow
Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
– Black
20 hours ago
add a comment |
(Reading the comments, it seems that you're trying to create a way to easily and reliably synchronize your production database to your staging site. Perhaps you could update the question to make this more clear.)
This is a common problem and having a tool to do this reliably is really beneficial. I have written my own bash script to do it and this is the basic process you should be trying to replicate:
- Dump the production database to a gzip tarball
- Download the dumped tarball to your staging server
- Drop and recreate the staging DB
- Import the production dump into the staging DB
- Adjust any configuration as appropriate for your staging server (mainly changes in core_config_data such as web/unsecure/base_url and web/secure/base_url etc)
I would suggest as a first step to get this down to a set of commands that you can reliably run to give you the desired result. Then once you have the process refined you can think about automating it with a simple script.
The tool that is really going to help you here is n98-magerun which is a priceless CLI tool for the Magento developer.
Breaking down the steps above into n98-magerun commands:
1) Dump the production database to a gzip tarball
On your production server:
n98-magerun db:dump -c "gzip" ~/latest-production-db.sql.gz;
Consider also using the --strip
option to remove sensitive data such as customer and order records. Depends on your use-case. See n98-magerun docs for Stripped Database Dumps.
2) Download the dumped tarball to your staging server
On your staging server:
Use rsync or scp or whatever CLI tool is most convenient or familiar to download the tarball created above.
Example using scp:
scp user@production:~/latest-production-db.sql.gz ~/latest-production-db.sql.gz
3) Drop and recreate the staging DB
On your staging server:
n98-magerun db:drop && n98-magerun db:create
4) Import the production dump into the staging DB
On your staging server:
n98-magerun db:import -c "gzip" ~/latest-production-db.sql.gz
5) Adjust any configuration as appropriate for your staging server
On your staging server:
n98-magerun config:set web/secure/base_url https://staging.domain.tld/
n98-magerun config:set web/unsecure/base_url https://staging.domain.tld/
Use the same config:set
command for any other configs that need setting. Note that config:set
can be scoped to a store or view. See the n98-magerun readme. Once you start to automate this process I highly recommend the HarrisStreet-ImpEx module for n98-magerun for managing your environment configs.
Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
– Black
17 hours ago
Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
– Black
17 hours ago
1
You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
– BlueC
17 hours ago
1
You could also use--include
on then98-magerun db:dump
command to explicitly include only the tables you want. Seen98-magerun db:dump --help
– BlueC
17 hours ago
1
I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
– BlueC
16 hours ago
|
show 1 more comment
We've got a similar setup and scenario. We went with a database level sync of product based tables. It works by dumping and importing tables.
The copy site is down for about 15 mins in early hours.
It was trial and error over which tables to sync.
Watch for eav tables though. There is a table which stores increment ids for orders, invoices etc. You dont want to sync this.
It was a very bespoke scenario and that worked for us. That's about as much info as I can give you at this stage. Good luck.
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "479"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmagento.stackexchange.com%2fquestions%2f278258%2fhow-to-keep-products-synchronized-in-two-shops%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
not sure how your copy store works, but you can use many options:
Magento Sync Products paid extension- Odoo connector Example free connector/ Odoo bridge Example Webkul paid extension
Magmi Magento Mass Importer- Store manager for magento
- Magento Dataflow
Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
– Black
20 hours ago
add a comment |
not sure how your copy store works, but you can use many options:
Magento Sync Products paid extension- Odoo connector Example free connector/ Odoo bridge Example Webkul paid extension
Magmi Magento Mass Importer- Store manager for magento
- Magento Dataflow
Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
– Black
20 hours ago
add a comment |
not sure how your copy store works, but you can use many options:
Magento Sync Products paid extension- Odoo connector Example free connector/ Odoo bridge Example Webkul paid extension
Magmi Magento Mass Importer- Store manager for magento
- Magento Dataflow
not sure how your copy store works, but you can use many options:
Magento Sync Products paid extension- Odoo connector Example free connector/ Odoo bridge Example Webkul paid extension
Magmi Magento Mass Importer- Store manager for magento
- Magento Dataflow
answered yesterday
MagenXMagenX
2,4171027
2,4171027
Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
– Black
20 hours ago
add a comment |
Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
– Black
20 hours ago
Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
– Black
20 hours ago
Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
– Black
20 hours ago
add a comment |
(Reading the comments, it seems that you're trying to create a way to easily and reliably synchronize your production database to your staging site. Perhaps you could update the question to make this more clear.)
This is a common problem and having a tool to do this reliably is really beneficial. I have written my own bash script to do it and this is the basic process you should be trying to replicate:
- Dump the production database to a gzip tarball
- Download the dumped tarball to your staging server
- Drop and recreate the staging DB
- Import the production dump into the staging DB
- Adjust any configuration as appropriate for your staging server (mainly changes in core_config_data such as web/unsecure/base_url and web/secure/base_url etc)
I would suggest as a first step to get this down to a set of commands that you can reliably run to give you the desired result. Then once you have the process refined you can think about automating it with a simple script.
The tool that is really going to help you here is n98-magerun which is a priceless CLI tool for the Magento developer.
Breaking down the steps above into n98-magerun commands:
1) Dump the production database to a gzip tarball
On your production server:
n98-magerun db:dump -c "gzip" ~/latest-production-db.sql.gz;
Consider also using the --strip
option to remove sensitive data such as customer and order records. Depends on your use-case. See n98-magerun docs for Stripped Database Dumps.
2) Download the dumped tarball to your staging server
On your staging server:
Use rsync or scp or whatever CLI tool is most convenient or familiar to download the tarball created above.
Example using scp:
scp user@production:~/latest-production-db.sql.gz ~/latest-production-db.sql.gz
3) Drop and recreate the staging DB
On your staging server:
n98-magerun db:drop && n98-magerun db:create
4) Import the production dump into the staging DB
On your staging server:
n98-magerun db:import -c "gzip" ~/latest-production-db.sql.gz
5) Adjust any configuration as appropriate for your staging server
On your staging server:
n98-magerun config:set web/secure/base_url https://staging.domain.tld/
n98-magerun config:set web/unsecure/base_url https://staging.domain.tld/
Use the same config:set
command for any other configs that need setting. Note that config:set
can be scoped to a store or view. See the n98-magerun readme. Once you start to automate this process I highly recommend the HarrisStreet-ImpEx module for n98-magerun for managing your environment configs.
Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
– Black
17 hours ago
Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
– Black
17 hours ago
1
You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
– BlueC
17 hours ago
1
You could also use--include
on then98-magerun db:dump
command to explicitly include only the tables you want. Seen98-magerun db:dump --help
– BlueC
17 hours ago
1
I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
– BlueC
16 hours ago
|
show 1 more comment
(Reading the comments, it seems that you're trying to create a way to easily and reliably synchronize your production database to your staging site. Perhaps you could update the question to make this more clear.)
This is a common problem and having a tool to do this reliably is really beneficial. I have written my own bash script to do it and this is the basic process you should be trying to replicate:
- Dump the production database to a gzip tarball
- Download the dumped tarball to your staging server
- Drop and recreate the staging DB
- Import the production dump into the staging DB
- Adjust any configuration as appropriate for your staging server (mainly changes in core_config_data such as web/unsecure/base_url and web/secure/base_url etc)
I would suggest as a first step to get this down to a set of commands that you can reliably run to give you the desired result. Then once you have the process refined you can think about automating it with a simple script.
The tool that is really going to help you here is n98-magerun which is a priceless CLI tool for the Magento developer.
Breaking down the steps above into n98-magerun commands:
1) Dump the production database to a gzip tarball
On your production server:
n98-magerun db:dump -c "gzip" ~/latest-production-db.sql.gz;
Consider also using the --strip
option to remove sensitive data such as customer and order records. Depends on your use-case. See n98-magerun docs for Stripped Database Dumps.
2) Download the dumped tarball to your staging server
On your staging server:
Use rsync or scp or whatever CLI tool is most convenient or familiar to download the tarball created above.
Example using scp:
scp user@production:~/latest-production-db.sql.gz ~/latest-production-db.sql.gz
3) Drop and recreate the staging DB
On your staging server:
n98-magerun db:drop && n98-magerun db:create
4) Import the production dump into the staging DB
On your staging server:
n98-magerun db:import -c "gzip" ~/latest-production-db.sql.gz
5) Adjust any configuration as appropriate for your staging server
On your staging server:
n98-magerun config:set web/secure/base_url https://staging.domain.tld/
n98-magerun config:set web/unsecure/base_url https://staging.domain.tld/
Use the same config:set
command for any other configs that need setting. Note that config:set
can be scoped to a store or view. See the n98-magerun readme. Once you start to automate this process I highly recommend the HarrisStreet-ImpEx module for n98-magerun for managing your environment configs.
Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
– Black
17 hours ago
Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
– Black
17 hours ago
1
You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
– BlueC
17 hours ago
1
You could also use--include
on then98-magerun db:dump
command to explicitly include only the tables you want. Seen98-magerun db:dump --help
– BlueC
17 hours ago
1
I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
– BlueC
16 hours ago
|
show 1 more comment
(Reading the comments, it seems that you're trying to create a way to easily and reliably synchronize your production database to your staging site. Perhaps you could update the question to make this more clear.)
This is a common problem and having a tool to do this reliably is really beneficial. I have written my own bash script to do it and this is the basic process you should be trying to replicate:
- Dump the production database to a gzip tarball
- Download the dumped tarball to your staging server
- Drop and recreate the staging DB
- Import the production dump into the staging DB
- Adjust any configuration as appropriate for your staging server (mainly changes in core_config_data such as web/unsecure/base_url and web/secure/base_url etc)
I would suggest as a first step to get this down to a set of commands that you can reliably run to give you the desired result. Then once you have the process refined you can think about automating it with a simple script.
The tool that is really going to help you here is n98-magerun which is a priceless CLI tool for the Magento developer.
Breaking down the steps above into n98-magerun commands:
1) Dump the production database to a gzip tarball
On your production server:
n98-magerun db:dump -c "gzip" ~/latest-production-db.sql.gz;
Consider also using the --strip
option to remove sensitive data such as customer and order records. Depends on your use-case. See n98-magerun docs for Stripped Database Dumps.
2) Download the dumped tarball to your staging server
On your staging server:
Use rsync or scp or whatever CLI tool is most convenient or familiar to download the tarball created above.
Example using scp:
scp user@production:~/latest-production-db.sql.gz ~/latest-production-db.sql.gz
3) Drop and recreate the staging DB
On your staging server:
n98-magerun db:drop && n98-magerun db:create
4) Import the production dump into the staging DB
On your staging server:
n98-magerun db:import -c "gzip" ~/latest-production-db.sql.gz
5) Adjust any configuration as appropriate for your staging server
On your staging server:
n98-magerun config:set web/secure/base_url https://staging.domain.tld/
n98-magerun config:set web/unsecure/base_url https://staging.domain.tld/
Use the same config:set
command for any other configs that need setting. Note that config:set
can be scoped to a store or view. See the n98-magerun readme. Once you start to automate this process I highly recommend the HarrisStreet-ImpEx module for n98-magerun for managing your environment configs.
(Reading the comments, it seems that you're trying to create a way to easily and reliably synchronize your production database to your staging site. Perhaps you could update the question to make this more clear.)
This is a common problem and having a tool to do this reliably is really beneficial. I have written my own bash script to do it and this is the basic process you should be trying to replicate:
- Dump the production database to a gzip tarball
- Download the dumped tarball to your staging server
- Drop and recreate the staging DB
- Import the production dump into the staging DB
- Adjust any configuration as appropriate for your staging server (mainly changes in core_config_data such as web/unsecure/base_url and web/secure/base_url etc)
I would suggest as a first step to get this down to a set of commands that you can reliably run to give you the desired result. Then once you have the process refined you can think about automating it with a simple script.
The tool that is really going to help you here is n98-magerun which is a priceless CLI tool for the Magento developer.
Breaking down the steps above into n98-magerun commands:
1) Dump the production database to a gzip tarball
On your production server:
n98-magerun db:dump -c "gzip" ~/latest-production-db.sql.gz;
Consider also using the --strip
option to remove sensitive data such as customer and order records. Depends on your use-case. See n98-magerun docs for Stripped Database Dumps.
2) Download the dumped tarball to your staging server
On your staging server:
Use rsync or scp or whatever CLI tool is most convenient or familiar to download the tarball created above.
Example using scp:
scp user@production:~/latest-production-db.sql.gz ~/latest-production-db.sql.gz
3) Drop and recreate the staging DB
On your staging server:
n98-magerun db:drop && n98-magerun db:create
4) Import the production dump into the staging DB
On your staging server:
n98-magerun db:import -c "gzip" ~/latest-production-db.sql.gz
5) Adjust any configuration as appropriate for your staging server
On your staging server:
n98-magerun config:set web/secure/base_url https://staging.domain.tld/
n98-magerun config:set web/unsecure/base_url https://staging.domain.tld/
Use the same config:set
command for any other configs that need setting. Note that config:set
can be scoped to a store or view. See the n98-magerun readme. Once you start to automate this process I highly recommend the HarrisStreet-ImpEx module for n98-magerun for managing your environment configs.
answered 17 hours ago
BlueCBlueC
604515
604515
Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
– Black
17 hours ago
Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
– Black
17 hours ago
1
You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
– BlueC
17 hours ago
1
You could also use--include
on then98-magerun db:dump
command to explicitly include only the tables you want. Seen98-magerun db:dump --help
– BlueC
17 hours ago
1
I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
– BlueC
16 hours ago
|
show 1 more comment
Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
– Black
17 hours ago
Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
– Black
17 hours ago
1
You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
– BlueC
17 hours ago
1
You could also use--include
on then98-magerun db:dump
command to explicitly include only the tables you want. Seen98-magerun db:dump --help
– BlueC
17 hours ago
1
I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
– BlueC
16 hours ago
Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
– Black
17 hours ago
Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
– Black
17 hours ago
Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
– Black
17 hours ago
Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
– Black
17 hours ago
1
1
You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
– BlueC
17 hours ago
You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
– BlueC
17 hours ago
1
1
You could also use
--include
on the n98-magerun db:dump
command to explicitly include only the tables you want. See n98-magerun db:dump --help
– BlueC
17 hours ago
You could also use
--include
on the n98-magerun db:dump
command to explicitly include only the tables you want. See n98-magerun db:dump --help
– BlueC
17 hours ago
1
1
I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
– BlueC
16 hours ago
I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
– BlueC
16 hours ago
|
show 1 more comment
We've got a similar setup and scenario. We went with a database level sync of product based tables. It works by dumping and importing tables.
The copy site is down for about 15 mins in early hours.
It was trial and error over which tables to sync.
Watch for eav tables though. There is a table which stores increment ids for orders, invoices etc. You dont want to sync this.
It was a very bespoke scenario and that worked for us. That's about as much info as I can give you at this stage. Good luck.
add a comment |
We've got a similar setup and scenario. We went with a database level sync of product based tables. It works by dumping and importing tables.
The copy site is down for about 15 mins in early hours.
It was trial and error over which tables to sync.
Watch for eav tables though. There is a table which stores increment ids for orders, invoices etc. You dont want to sync this.
It was a very bespoke scenario and that worked for us. That's about as much info as I can give you at this stage. Good luck.
add a comment |
We've got a similar setup and scenario. We went with a database level sync of product based tables. It works by dumping and importing tables.
The copy site is down for about 15 mins in early hours.
It was trial and error over which tables to sync.
Watch for eav tables though. There is a table which stores increment ids for orders, invoices etc. You dont want to sync this.
It was a very bespoke scenario and that worked for us. That's about as much info as I can give you at this stage. Good luck.
We've got a similar setup and scenario. We went with a database level sync of product based tables. It works by dumping and importing tables.
The copy site is down for about 15 mins in early hours.
It was trial and error over which tables to sync.
Watch for eav tables though. There is a table which stores increment ids for orders, invoices etc. You dont want to sync this.
It was a very bespoke scenario and that worked for us. That's about as much info as I can give you at this stage. Good luck.
answered yesterday
Dominic XigenDominic Xigen
1,8101311
1,8101311
add a comment |
add a comment |
Thanks for contributing an answer to Magento Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmagento.stackexchange.com%2fquestions%2f278258%2fhow-to-keep-products-synchronized-in-two-shops%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
sounds like a use-case for using the multi-store capability of magento rather than having two separate instances that need to be synced.
– BlueC
Jun 14 at 12:37
@BlueC, thats not possible since both shops are using different code (different git branches checked out)
– Black
2 days ago
1
What's different about the code? Can you unify the changes using themes/child themes etc? Magento is built to do this.
– BlueC
2 days ago
1
Hmmm, templates and scripts can all be utilised per store quite easily. Extensions can be a pain because they can conflict. Observers could/should be written to be configurable per store view. I am not saying its impossible to synchronize the products, but it doesn't seem a very elegant solution, and longer term having a single store with unified products might be preferable. Does sound like it might be tricky to untangle though.
– BlueC
2 days ago
1
Reading below it seems you don't actually want or need 2 storefronts, but you just want a staging and production site and to be able to sync from production to staging. This wasn't clear in your question, and in this case using the multi-store setup in Magento is not appropriate.
– BlueC
19 hours ago