Will using async or bulk endpoints actually improve performance of import processes?Pinterest Buyable Pins Magento IntegrationMagento 2 REST API checkout payment gateway redirectCaching / invalidating cached responses to Magento 2's REST API?Connect Magento site with Php site?Use API rest media management by storeView code (admin)Improve performance on SOAP callsMagento 2 - Routers_Match slow?First steps of creating API integration with Magento2.3Do I really need Magento over WooCommerce

Shift lens vs move body?

Does a Mace of Disruption's Frightened effect override some undead's immunity to the Frightened condition?

Changing JPEG to RAW to use on Lightroom?

Is it ok to record the 'environment' around my workplace?

Biological refrigeration?

Which old Technic set included large yellow motor?

Papers on arXiv solving the same problem at the same time

Is a memoized pure function itself considered pure?

Who was the most successful German spy against Great Britain in WWII, from the contemporary German perspective?

Unlock your Lock

How can I download a file from a host I can only SSH to through another host?

Why is sh (not bash) complaining about functions defined in my .bashrc?

Talk interpreter

Can MuseScore be used programmatically?

Notice period 60 days but I need to join in 45 days

How many birds in the bush?

How does the OS tell whether an "Address is already in use"?

What's the point of fighting monsters in Zelda BoTW?

What's special ammo in Destiny 2?

1mth baby boy keeps peeing through diapers, sometimes diper seems practically unused

Expressing an implication as ILP where each implication term comprises a chain of boolean ORs

LINQ for generating all possible permutations

To what extent should we fear giving offense?

What are the IPSE’s, the ASPE’s, the FRIPSE’s and the GRIPSE’s?



Will using async or bulk endpoints actually improve performance of import processes?


Pinterest Buyable Pins Magento IntegrationMagento 2 REST API checkout payment gateway redirectCaching / invalidating cached responses to Magento 2's REST API?Connect Magento site with Php site?Use API rest media management by storeView code (admin)Improve performance on SOAP callsMagento 2 - Routers_Match slow?First steps of creating API integration with Magento2.3Do I really need Magento over WooCommerce






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








0















We have a multi-store setup with 16 stores and every store has it's own language, we have a lot of store-specific information. Most of our products are configurable products with lots of variants, so all in all we have about 6000 simple or configurable products.



We have a middleware in place that manages upload of categories, product and media via Magento2's synchronous REST API. This middleware has its own message queues and concurrency management. I played a bit around and ended up with about 5 concurrent workers as the best tradeoff between performance and Magento-deadlocks.



In the rare case that I have to upload all products, I have to perform about 6000 products * 16 stores = ~ 100.000 API calls which takes a lot of time, eventough our instance is running on an AWS t2.2xlarge instances. Usually it takes more then 24 hours.



I understand how utilizing bulk or asynchronous APIs can facilitate and optimize things when you don't have a message queue in place yet. But do you think switching over to async or bulk APIs could still signifantly optimize performance compared to our solution? I don't want to optimize just the time it takes to hand over a task to Magento but I want to optimize the time it takes a change actually is applied and becomes visible for the customer. If Magento's bulk/async APIs use the exact same internal APIs as the sync-API, I don't see any potential optimizations except saving network latency here. What do you think?



The management interface of our middleware










share|improve this question
























  • Any experiences?

    – derspringendepunk
    Aug 21 at 8:53

















0















We have a multi-store setup with 16 stores and every store has it's own language, we have a lot of store-specific information. Most of our products are configurable products with lots of variants, so all in all we have about 6000 simple or configurable products.



We have a middleware in place that manages upload of categories, product and media via Magento2's synchronous REST API. This middleware has its own message queues and concurrency management. I played a bit around and ended up with about 5 concurrent workers as the best tradeoff between performance and Magento-deadlocks.



In the rare case that I have to upload all products, I have to perform about 6000 products * 16 stores = ~ 100.000 API calls which takes a lot of time, eventough our instance is running on an AWS t2.2xlarge instances. Usually it takes more then 24 hours.



I understand how utilizing bulk or asynchronous APIs can facilitate and optimize things when you don't have a message queue in place yet. But do you think switching over to async or bulk APIs could still signifantly optimize performance compared to our solution? I don't want to optimize just the time it takes to hand over a task to Magento but I want to optimize the time it takes a change actually is applied and becomes visible for the customer. If Magento's bulk/async APIs use the exact same internal APIs as the sync-API, I don't see any potential optimizations except saving network latency here. What do you think?



The management interface of our middleware










share|improve this question
























  • Any experiences?

    – derspringendepunk
    Aug 21 at 8:53













0












0








0








We have a multi-store setup with 16 stores and every store has it's own language, we have a lot of store-specific information. Most of our products are configurable products with lots of variants, so all in all we have about 6000 simple or configurable products.



We have a middleware in place that manages upload of categories, product and media via Magento2's synchronous REST API. This middleware has its own message queues and concurrency management. I played a bit around and ended up with about 5 concurrent workers as the best tradeoff between performance and Magento-deadlocks.



In the rare case that I have to upload all products, I have to perform about 6000 products * 16 stores = ~ 100.000 API calls which takes a lot of time, eventough our instance is running on an AWS t2.2xlarge instances. Usually it takes more then 24 hours.



I understand how utilizing bulk or asynchronous APIs can facilitate and optimize things when you don't have a message queue in place yet. But do you think switching over to async or bulk APIs could still signifantly optimize performance compared to our solution? I don't want to optimize just the time it takes to hand over a task to Magento but I want to optimize the time it takes a change actually is applied and becomes visible for the customer. If Magento's bulk/async APIs use the exact same internal APIs as the sync-API, I don't see any potential optimizations except saving network latency here. What do you think?



The management interface of our middleware










share|improve this question














We have a multi-store setup with 16 stores and every store has it's own language, we have a lot of store-specific information. Most of our products are configurable products with lots of variants, so all in all we have about 6000 simple or configurable products.



We have a middleware in place that manages upload of categories, product and media via Magento2's synchronous REST API. This middleware has its own message queues and concurrency management. I played a bit around and ended up with about 5 concurrent workers as the best tradeoff between performance and Magento-deadlocks.



In the rare case that I have to upload all products, I have to perform about 6000 products * 16 stores = ~ 100.000 API calls which takes a lot of time, eventough our instance is running on an AWS t2.2xlarge instances. Usually it takes more then 24 hours.



I understand how utilizing bulk or asynchronous APIs can facilitate and optimize things when you don't have a message queue in place yet. But do you think switching over to async or bulk APIs could still signifantly optimize performance compared to our solution? I don't want to optimize just the time it takes to hand over a task to Magento but I want to optimize the time it takes a change actually is applied and becomes visible for the customer. If Magento's bulk/async APIs use the exact same internal APIs as the sync-API, I don't see any potential optimizations except saving network latency here. What do you think?



The management interface of our middleware







magento2 api rest






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Aug 14 at 15:02









derspringendepunkderspringendepunk

1




1















  • Any experiences?

    – derspringendepunk
    Aug 21 at 8:53

















  • Any experiences?

    – derspringendepunk
    Aug 21 at 8:53
















Any experiences?

– derspringendepunk
Aug 21 at 8:53





Any experiences?

– derspringendepunk
Aug 21 at 8:53










0






active

oldest

votes














Your Answer








StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "479"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmagento.stackexchange.com%2fquestions%2f285512%2fwill-using-async-or-bulk-endpoints-actually-improve-performance-of-import-proces%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes















draft saved

draft discarded
















































Thanks for contributing an answer to Magento Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmagento.stackexchange.com%2fquestions%2f285512%2fwill-using-async-or-bulk-endpoints-actually-improve-performance-of-import-proces%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Grendel Contents Story Scholarship Depictions Notes References Navigation menu10.1093/notesj/gjn112Berserkeree

Log in Navigation menu

Invalid response line returned from server: HTTP/2 401 | ErrorPlease Please Help With Error 500 Internal Server Error after upgrading from 1.7 to 1.9Unable to place new customer orders in admin backendMagento - For “Manage Categories” Forbidden You do not have permission to access this documentHTTP ERROR 500 when using require(_once) app/Mage.phpMemcached causing Web Setup Wizard ErrorCould not create an acl object: Invalid XMLAn error occurred on the server. Please try to place the order againInvalid response line returned from server: HTTP/2 200 - message after update to 2.1.7Magento-CE 2.3.0 installation error on XamppMagento 2.2.6- After Migration all default Payment Methods are not working fine