Load In The Loop: Episode 7 – Southco
Load In the Loop: Episode 7 is here! In the first episode of 2020, join Eric Hileman and Ivan Chepurnyi as they identify and fix some performance issues for Southco.com.
Sponsored by blackfire: https://blackfire.io/
Follow us on Twitter! https://twitter.com/loadintheloop
Ivan: firstname.lastname@example.org, @IvanChepurnyi
Eric: email@example.com, @ericvhileman
Eric Hileman: [00:00:00] Load In The Loop is sponsored by Blackfire dot. IO I’m Eric Hileman, CEO and cofounder of MageMojo Magento hosting in New York. Purpose of this program is to take some of what we’ve learned from over 10 years of Magento hosting and evaluate some of our customers stores, and then share that with our audience, what we find and hope that it improves their debugging skills and performance skills, and hopefully extension vendors get at us and we can help make the extensions, especially the really popular ones, better for everybody. And I’m here with my cohost Ivan.
Ivan Chepurnyi: [00:00:37] Hello, my name is Ivan. This show, for me is a great chance to share. My experience that I’ve accumulated in Magento, area for the last, 12 years, almost 13 years already.
So, the first time when I met the guys for OMA Larian, I was in the winter of 2006. So it’s, it’s almost 13 years, since I, am, in LA was Magento and to magenta platform itself. So, I run a consultancy business here in . I help merchants begin small to solve their performance issues and the complexities of working with Magento.
So tell us, Eric, what. Date of customer we have today,
Eric Hileman: [00:01:23] dude, Kaylin. Kaylin posted a, a, a job or a curve calmer, a person for commerce here. And he said they have 13 years experience in Magento and selling reply. Like, how do you have 13 years experience with Magento? And I guess that’s how from clear back to 2006.
Ivan Chepurnyi: [00:01:39] So it was a South . So at that time, Magento. Watson worry, worry, early stage even, you know, not existing as a code,
Eric Hileman: [00:01:52] commerce or just pieces.
Ivan Chepurnyi: [00:01:55] Yeah. And kind of meetings. So when the team, was first time gathered together. in the and, Martin company was working on those core Morris projects at the time, and I was just invited to try ’em they could look at the, as a team, is it to work so it was there and to get to know people and check if we are good fit from point of view of possible collaboration as employee, employer relationship.
Yeah, let’s do a quick recap, over the last two episodes, right. So a lot of, people were giving feedback on the idea of all of our show. A lot of people really enjoy it to, and I really liked that to a lot of people actually like to, which, Awesome. Discover performance issues. It’s actually rises quiet.
The light, lively discussions on Twitter and the magenta chats and so on.
Eric Hileman: [00:02:56] Some engagement with extension vendors too. So there’s been some fixes that were released, and I the latest one, you were talking to eBiz marks and working through the config or the cash clearing on production.
Ivan Chepurnyi: [00:03:10] Yeah. It’s, it’s nice that they reacted to it, right away.
unfortunately they didn’t, read very carefully is a description of the issue because my main concern was not just, You know, the initiative be call, but mainly also storing the configuration of value on the front end and flushing the cash. So I hope they will reopens it. Do you get them to, we’ll continue working on it from perspective because this, is there a snow place for changing your configuration where elders, Sarah was recently actually a, If we threats threat as well, from, one of those, CTOs of one who was a merchants, let me look it up quickly. Yeah. Jonathan, every bus. So, he recently tweeted the bylaws. The issue, is that, he had to was, here’s a Magento store, is this, this is Attica and wall terror. it’s, one was a short case store, during magenta life, and they have this, extension, called, feed management from Lango.
And this extension was actually, saving some data into. Configuration table every hour. So they probably have some kind of a Cron job and they probably sort of sometimes stamp. What was the last, modified to ’em? Yeah. Feed or something. Or is this the last modified, or was the last time the food was getting ready as they progress.
Not great on table for this data, but they probably just decide, Hey, which I was going to story it to incorporate UNFI data because we already have it, and they use of course for that. Do they use standard magenta model for saving the a configuration values and not a lot of people know about this model actually completely wipes out a configuration cash.
And if had happens every hour. it means that, is this store I’m under high Lord. So if you’re out for black Friday sale or something and Zen every hour, dude in black Friday, sale your configuration, casual gun and get the nuked, you will receive a lot of issues because right now, magenta doesn’t have, a different in place from my pool requests that I was doing for configuration cash and locks.
And, yeah. As a problem was, is that Magento, right? Right. Now, does it extra curricular work? is it configuration during high concurrency? And it’s the same time, it has a lot of concurrent processes actually generating cash and blocking, FBM pool from processing or requests
Eric Hileman: [00:05:36] cash stampede.
Ivan Chepurnyi: [00:05:38] Yeah. And a support request. I’ve graded actually a souls this problem in a very interesting way, because instead of flushing the cache, what I do, first of all, every time when I get rate, the new cash was in, you catch prefix. So instead of a jazz deleting a record and the writing the new record. It actually keeps as the old record to steal in the system while it’s generates a new cash and Zen and justice.
We chose as a cash shut. There’s a new,
Eric Hileman: [00:06:08] that’s right. We were talking about that. You mentioned that in the zero downtime deploys, when we had some customers where we suspected there was Radis issues and putting a yeah, cash prefix into the E and V. dot. PHP when they were doing deploys.
Ivan Chepurnyi: [00:06:24] Is it, is this, this one thing was a retina deploys is actually somethings that I’m doing now for one of my, a magenta costumers.
I’d be able to, for them, some deployment pipeline where every new built artifact is going to have a unique cash Prophix. And it means that, Every single new code version will have a separate write discuss and they will never intersect and they will never overlap. So boost good versions can leave some mantis Lee as the same time as soon as database upgrade is not needed.
So this is quite nice for a. Gracefully rolling out the new magenta versions in the production. Yeah. Or, you know, like new functionality. And so, and so,
Eric Hileman: [00:07:06] have you had any feedback from, MSD or Mirez feat that say that right. His feet.
Ivan Chepurnyi: [00:07:12] I’ve, I think, like Zay never send me an email. I really liked to . On a wave, was that Zay war really, listening, to our repeat backends?
They even apply it to in your version. Unfortunately, we didn’t have the time to review it yet because we don’t have yet. The customer was this new version installed. but as soon as we have with customers that we were talking with some email, you remember, who are. Who, who wants to be in an show. And, as soon as we get these customer on our shows, and we can actually use this portion because he had the holds that
Eric Hileman: [00:07:48] we should do a recap show where we recap the customers and to show the changes that they made.
Cause they really. Then helped a lot. They’ve been pretty happy and so many extensions vendors are actually communicating and fixing things, which is awesome and be so, we want to talk to extension vendors. We don’t want to just call people out and point out problems like we want to help them fix them and give them advice.
So extension vendors shout out at us, well, we’ll work with you and help you as best we can.
Ivan Chepurnyi: [00:08:15] Yeah. And you know, the thing is that. A lot of extension renders Z just a probably doesn’t know that those probably wouldn’t exist because a Z now half a real beta obser real merchants a Wednesday actually build extensions and probably on a small data set, they’ll see shifts are completely invisible, for an extension of vendors.
Because when you have a small amount of data ever fitness fast.
Eric Hileman: [00:08:44] Yeah, yeah. Works. Works on my local development.
Really. You have all the data and you’re in your database, and is it, does it match production though? No, probably not.
Ivan Chepurnyi: [00:08:59] Yeah. And also things are sounding, should be with magenta core team as well. They also should develop, at least on a database with 20,000 products. They should never develop on a database with hundreds.
Right. So he just doesn’t make any sense.
Eric Hileman: [00:09:18] Yeah. Not really sure how that MSI release. Got out with those issues in it. And, you know, we see a lot of issues with chronic consumers too. Even even when we set them up, we still see issues with them. I, I don’t know, I dunno. I dunno. That’s raises questions that someone else can maybe answer about their automated testing and that’s going on for performance regressions.
Ivan Chepurnyi: [00:09:47] Yeah. It’s a.
They have probably some kind of protests, but I don’t think that they really test the real production, like a use age because for instance, a lot of people right now tells a, Hey, magenta one is much more fosters than magenta two kind of. Yes, because there is less bottlenecks in the magenta one, but there are still some issues exist in magenta.
One is that they were doing it for a very long time. or is it my crew year? And one was the main issues is actually, as soon as you have someone working in the admin panel, magenta one becomes really, heavy loaded. And what I saw, for instance, from magenta to perspective for the new indexation models that was introduced in magenta two is much more better from point of view of.
stability of the website. And if you change a lot of products or you have for your integrations, Magento two is much more better, at, not. Breaking down your frontend because magenta one a, as soon as you have 400 products being imported, at some point, Magento might, produce some deadlocks or even wipe out, complete a product list on the category.
And. Yeah, it just happens all the time in magenta one
Eric Hileman: [00:11:13] yeah. My, my team, my team here, they, we, we get into situations where customers are, they’ll have a big traffic event and then there’ll be working in the admin and it’ll trigger something in the admin that slows performance down. And my team from the backend is like, well, why.
Why would you be working in the admin when, you know, you just sent out this big email campaign, like leave it alone. But from the merchant perspective, they’re like, well, we’re watching this sale and we’re seeing things that we need to change. Like we want to merchandise, so we want to move this product here.
We want to, you know, change the color of that. We want to tweak things as we watched the sales go, and merchants want to do that stuff. So yeah, you know, you have to, you have to think about that and enable that
Ivan Chepurnyi: [00:11:55] is the interesting things that I’m actually. is there a lot of, nice implementations in magenta too, especially based on my feedback in a devised and 16, I was giving a lot of talks about indexation of how to build Phoenix cars properly.
And Magento too, a lot. It was OCD as and implemented actually as, in current most recent Magento version. So right now, every single index, you have a variety index. When there is a full re index happening, it’s always going to right. There’s a date that into a separate table, never into the live windows.
So basically you never have a situation where, you are getting rating, an index and all your products apparently are gone during this process. So, right. Right now, Magento switches a tables after indexation has done. And this is a, I think it was from my side in order to convince them, Hey, you need to change the way how you do it.
Eric Hileman: [00:12:56] Yeah. Merchants want to work on their store while they have stuff going on. Like you can’t, I mean, ideally we would say, okay, just queue all your workup for the end of the day and then release it at the, in the middle of the night or something. But that’s just not how. Merchants, ones the store,
Ivan Chepurnyi: [00:13:13] what two is magenta two.
It’s possible, and it’s very nice, is that it’s actually possible to do a with standard magenta, so you can just set all your indexes in to update on schedule. And it doesn’t matter how many updates you do, it will not do it. in one. After each update it, we’ll do it in a scope of batches of products being updated for Zendek.
So. Is there this last rebuilding? Yeah.
Eric Hileman: [00:13:45] We put that into our terms of service actually. We’re like, okay, you need to, you have to set your indexes to update on schedule. We don’t, we don’t actively like probe for it and enforce it, but when people come back to us and they’re like, but I need to see my product changes immediately.
Ivan Chepurnyi: [00:14:02] sorry.
Eric Hileman: [00:14:03] Like you can’t, you can’t be rotted all this traffic and to seeing all your changes immediately and stacking all your, your indexing changes. Like it’s just, you can’t do that.
Ivan Chepurnyi: [00:14:14] Yeah. I think a lot of ’em. March. And so for, for instance, I had merchants, in my career where merchants wanted to see changes immediately from European, from PIM, and now, and this is this much more harder because standard ways, how you import products and manipulate the data as well, considering that, for instance, from beam, you also need to download images and you need to set those images appropriately.
And in other ways, it was. Almost an insurable thoughts again to, I just explained to immersion. Yeah. If you want to have all your new collection being visible. let’s say it was in a minute, it’s not going to be possible because, your ERP server gives us very big images that we need to spend time to download SOC.
Yeah. Europe, European system. So yeah, it’s, it’s always, tried to explain, what actually happens during the process and why sometimes it takes a long time. However, right now with my experience, if I would have the same customer right now, is that I wanted to have something immediate, I already would have some solution for it.
So, right now I do a lot of, SyncroNis programming, and somethings that actually a lot of people confuse programming with concurrent programming. So it was in coronaries programming is when you, do a work on a CPU while, You are waiting for some IO to complete. So let’s say you are grabbed something from the network.
So you tell your told operation system, Hey, can you please, sends his data over this socket? And as soon as the data is back, give me an event and tell me as that, the data is already available and you can possess it. So this is the basic idea of his insurance programming that during the time while operating system does this stuff.
Oh, what was the network? at the same time, I can go on and do, Azara work myself. So for instance, I can ask, Hey, can you Deloitte 10 images for me and Zen is this 10 images getting downloaded a concurrent of concurrently, because I do sequential start because I have one with single thread, but, Zara downloaded the concurrently from point of view is that I don’t need to wait, for each image to download a by itself. I can just go on and request request. Nasry mentioned Nazarene mentioned another image. And then the Wednesday images just are all downloaded. I can just insert records into the database.
And this is central honors programming and a lot of people are confused and synchronous programming was using message queue where you just push some message and does a message queue and Zen, you have Justin of receives this message and does standard two blocking operations. So is this his goal to. not, I see furnace programming.
It’s called to more kind of background processing with current workers. Yeah. You, you, you, you just use the same words that you would have done in the main, process while positions or request, comparing, just to be able to actually do other work while you wait for. are you? So,
Eric Hileman: [00:17:27] yeah. Yeah. We, we see that.
I don’t, I, we shouldn’t get too deep into it, but we do see that when we look at performance metrics, CPU time will be used, but then we’ll look and we’ll be like, ah, nah, it’s just hanging out waiting on IO. Oh wait. Like, it’s not actually using the core, it’s just sitting there waiting, even though it looks like it’s in use.
So a Southco so I don’t think it’s, I don’t think it’s live yet. They’re still testing it. well, yeah, let’s just jump into it and see what we find. I’m not familiar with all of the,
Ivan Chepurnyi: [00:17:58] okay. So, so if it’s like a, okay. Differing locks. Lecherous okay, so Zay probably a manufacturer stuff and maybe even on demand because as they see, you can download CAD files, guide files, and so on. So interesting. Okay, so let’s disable aggregation right away, and let’s run a profiling.
So is that to a squeak for a homepage right now? Not too bad. Well, let’s see what, Information that the way square is full of Dallas. No, I kinda
Eric Hileman: [00:18:47] liked this design with the red blue, the red blueprints in the back. It’s kind of neat.
Ivan Chepurnyi: [00:18:54] Yeah, it’s, it’s very, nice and so, yeah. Okay. So we have 1.3 seconds, and probably is actual time is around 800 milliseconds.
If we are, we do it without Blackfire. I already see someone’s, the problems right away here. if take a look, you can see sorted by sequence and during the last show, you remember we had, one those, extensions accessing module version from configurations that was in walk-ins. This sword by sequence.
So let’s switch right away into the, timeline of view and see where, where it happens here. So here it is. Search forum, GS. And this is collab a module. So glad was search, does the some, module information fetch in order to know probably which module version is installed. So again, does the same thing as it would discuss in the previous show.
you don’t need to use this adopt portion from XML. You better create a class in your module. and debated was every single Ruiz, so was a constant of your module version.
Eric Hileman: [00:20:14] Just
Ivan Chepurnyi: [00:20:14] might get easier for a. Magento don’t ask it through a configuration retrieval. It doesn’t make sense.
Eric Hileman: [00:20:22] Yeah. You already know it.
Just bump it when you release.
Ivan Chepurnyi: [00:20:25] And especially in us, our important thing is that a lot of people don’t know when you update the stock version in Magento, Magento is forced to, run a set up upgrade. So you cannot. Zen deploy, was our downtime. You cannot deploy a new version of this extension to a production.
And if your extension and they’re going to expect that a cloud or is an extensions that doesn’t store anything, does a database, I don’t think that’s a hour for a set up upgrades. running as a way to modify as their schema. And as well, we can 2.3 there is already declarative schema and notations at the soil level.
So it’s completely doesn’t make sense to use as this, module version as a version of the code. Because Zen, it means that with every single code update, or you would trigger a Magento, that the way school rate as well is that it’s completely unnecessary if you don’t have any of that, the race of grades.
So please remove from your modules. If you don’t have any, set upper grades, please remove for your memorials is the top portion completely. and what’s a restriction on 2.3 and higher. in utero quarter base where this, attribute is removed, it will play a very good to favor for the emergents around Zara is that try to reduce as erudite downtime when they release important updates of their stores.
Because one more less, modules that, bumps worship during gold changes in a XML file. So is this, this is basically a thanks, a free a hundred millisecond from 1.0 to seven so it’s already one force of the Lord time.
Eric Hileman: [00:22:18] now, would that be better if they optimized? composer, we, I tweeted that out. and the link to it, we talked about that last show where composer wasn’t optimized.
If composers optimizes that check quicker.
Ivan Chepurnyi: [00:22:33] yeah. no Nazis. Fun. So is this one is actually a very slow process because Magento performs a graph, sort. So the graph sort, it’s a kind of a sort where you. don’t have a sorter, but you have information. Hey, this module goes with Ford as this one.
Would you all buds is all goes before is that module. So you kind of cannot sort it very efficiently without, you know, doing a lot of a redundant operation. So that’s why sort by sequence takes so much time. And in other thing is that there’s important, is that modules get all. It actually loads all the configuration files of all of the modules available in the system.
So apart from a actually slow process, it also, it’s quite a lot of, memory for every single PHP requests out there. Because the six among used to be loaded into the memory and basically Magento itself doesn’t use this module version anywhere in own modules and to, it only uses it during a setup upgrade operation when it needs to check.
What does the current version define an XML and what does it call integration defined in that way. Stable. And it was a Diefer Zen. It, it could give us an update.
so let’s see, what else do we have here?
Also. Everything else seems good. So I don’t see somethings that you know, stands out here. So main problem here is, is this global search forum
Okay, so, so, so, so Zay also have magenta commerce, right? Installed. So they’re the commerce customers. Okay. Let’s, go.
Eric Hileman: [00:24:39] Products.
Ivan Chepurnyi: [00:24:40] Let’s go to CMS page. It’s interesting because is it first the companies that I see zip boots, into the top menu? The first thing is about us. So interesting to profile their about us page.
It’s, it’s very interesting, from point of view. It was a commerce store because I never see any eCommerce stores that, cares about their brand so much. So it’s very nice. And especially with our history. Yeah. It looks very nice.
Eric Hileman: [00:25:06] Does
Ivan Chepurnyi: [00:25:09] beautiful. Oh, Oh, good. So let’s see. What’s the result for a homepage?
Oh, sorry. Not the homepage. Same as page.
Oh, okay. so yeah, again, probably the same thing. Oh, definitely. Search form. So yeah, CMS page, it looks good. probably not much to optimize. The only things that, is interesting here is actually is, is fun by, this is the standard Magento issue. and this issue, as far as I remember, I was fixing for wrongs and merchants by adding an override around the, Around to edit through to select. So the problem is when, let’s say you tried to load a product collection and the inside was this product collection. Magenta has small performance issue where if you are not using flat catalog cause it is actually duplicated. Now in magenta, right? So you shouldn’t use flat catalog.
And when you follow this practice, when you don’t use flat catalog, is there a problem? Is that every single attribute you want the Lord within the collection gets loaded by magenta separately instead of doing it for the collection of attributes? So let’s say you, if you have, 50 attributes, to be evolved within the collection.
My will load every single of these 50 attributes separately, and then you will see more queries here. So this greater, is it busy here? This is, are directly a result of that, get attribute to by code, from EAV model.
Zoe, if you’d like to fix it local in your own project, you can add a plugin around at the root to select for the product collection and then just brief your pre-populate as the attributes for a UAV config by using a functionality available. It was there. Poor population. I was always had their abuse, so, okay, let’s go on into another page.
Right. Well, I had to go to products. let’s go to the main category. Gold latches, right? So this is a category of categories as far as AC. So let’s, profile, is it.
Eric Hileman: [00:27:41] I never thought there was so many latches, latches, rotary latches, push to close latches,
Ivan Chepurnyi: [00:27:50] you know, Oh, the world is alleged on the lecherous.
Okay, let’s see what’s going to be the result.
Okay. I think we will see the same thing. So here we don’t see any award in the loop for a category. So again, it’s the same thing as before. The main contributor is still collateral. So it seems like a, this merchant put a lot of effort into optimizing the store already.
Eric Hileman: [00:28:32] Yeah. They’ve been working on it.
Ivan Chepurnyi: [00:28:35] Yeah. So let’s take a look at a push to latches. Push to quote slouches. Okay. Is this page was loading a little bit slow. So pro is, there is some, complex functionality here, and I already see quite complex speech.
Eric Hileman: [00:28:51] There’s a lot going on on that page.
Ivan Chepurnyi: [00:28:53] Wow. Yeah. So this is kind of interesting. Lard navigation style.
So let’s do the blow corrugation, because I think we’re going to see some basic waivers here.
Eric Hileman: [00:29:24] Okay.
Ivan Chepurnyi: [00:29:37] Oh my goodness. A one 1,750 that the waste graders. Okay, Eric.
Eric Hileman: [00:29:55] That’s about what I expected. There’s a lot going on in that page. Like
Ivan Chepurnyi: [00:30:00] ads, they have so many filters, mitigation as well, so we’re always there. A social South slope cars too.
So let’s take a look at a timeline. What’s taking the most time on a page? So broad ducks, they can rent it and it was a BoardEx stakes, three seconds and.
Okay. The biggest time was actually spent
building actually is those segregations you see here. So buildings are already, navigation is the most, consuming, task on the product. least. So Zay probably optimized very well. Zay optimized the product list itself because if you take a look at the product, the list template, it takes almost napping.
So. actually here we can see a one of the standard Magento, performance bottlenecks and sufferings that they pick souls the time. It’s a missing cash around the configuration of you. So this functionality, for configuration of you, if you tried to access any media attribute of the product. magenta tries to find the configuration values for this attribute, for resizing the Emailage inside of, view configuration and few configuration is located in every single Azeem and net, and also kind of go get it in catalog module.
So what it does here, it merges all the files together and determines the result. And this result is actually uncashed. So Magento never caches information retreat from this, model. And what I do, I just add the plugin usually to this root method, is that gaseous output of the wreath massive. Into the cash storage and Zana we are good to go without this IO call on every single page.
Eric Hileman: [00:32:19] Yeah, we see that a lot. Checking image attributes on every car.
Ivan Chepurnyi: [00:32:25] Yeah, so it’s like weeks, weeks four is that one is just to add the caching on top of it through a plugin. So this is a a problem. Does the only problem in product list itself,
Eric Hileman: [00:32:39] there’s no extension out there that does that, that caches the
Ivan Chepurnyi: [00:32:44] not yet.
I probably will release it
Eric Hileman: [00:32:46] because
Ivan Chepurnyi: [00:32:49] I have one already is a digest install, but it’s not, some things that they consider an extension is that this like makes sense to release this. An extension itself because the just couple of lines of code, a in configuration and a couple of flights of golden PHP file.
So not much going on there
Eric Hileman: [00:33:12] is
Ivan Chepurnyi: [00:33:12] that may basically extension with single plugin.
Eric Hileman: [00:33:16] Can we, can we link that? Do you have like a just or any, any links that we can .
Ivan Chepurnyi: [00:33:22] I think I will definitely put it on GitHub. Okay. So here we have layer, view, beach, GML. here’s some stuff going on. All right, so this is probably, some stuff that gets a render it in a loop.
And what happens here, it’s actually. Does it do any IO here? Yeah, actually it does 404 milliseconds of radio, but I unfortunately, I don’t see what kind of eye operations are happening there. Oh, here it is. Count, count, count, count, count, count. So is there a lot of account operations for every single filter.
In order to count the number of products with specific attribute value set. So first of all, what I can advise for this customer, try and to use, elastic. Cute. So right now this merchant actually using a Magento, standard, my ischial alert navigation. Is that this wary, wary, slow, and produce a Squires like that.
And square is just a eat up all of your server sources because magenta creates a lot of temporary tables. And if you have high concurrency, it may resolve in where we, overloaded my sequel. is that, we’ll constantly, bottleneck on IO because memory tables are only created the ma, sorry.
Temporary tables are all created in memory. If there is enough memory for those memory tables, and when you have too much memory tables, majority of Sam will just be creative on these. And Zen, every single gration on disc is going to be painfully slow. So.
What is print arguments? I never seen this one
Eric Hileman: [00:35:26] either.
Ivan Chepurnyi: [00:35:31] And it’s like one or seven times, so weird one. So definitely they need to look what is it, what it is. And as usual, I see the standard issues here. product category gets loaded 51 time on the page. So I think for every single, Oh wait, it’s Magento commerce right?
Eric Hileman: [00:35:56] Yeah.
Ivan Chepurnyi: [00:35:57] So yeah, it’s a standard problem of Magento commerce and Magento commerce catalog events are a, adding some observers on product collection Lords, and it tries to find the appropriate gotta look around for a product in order to, show, you know, is this countdown, dimers that bell, Hey, you have special offer.
So. The easiest fix to remove all of this square is related to . Integrities is actually a disabled couple of events or fix a module itself. So fixing the module is much more harder. Disabling ketorolac events, if you don’t use them, is much more easier. Right? So it was a customer doesn’t, through scuttle Oak events.
Disable it. and the other one is the standard problem with, gash, tagging. Is it the Magento has, when you have our initial enabled. Huh. So we’ve seen this on the previous episode and we also see it in a script. Gore as well. So this is an issue is that it’s possible to fix by disabling original plugins and creating just important, probably, I’m going to release this module as open sources that specifically fixes this huge performance bottleneck because this is like 160 choirs where you have a lot of products.
So every single product that is shown on the page gets this query executed. That’s completely. Not needed. Here we have, again, attributes and the, this time it’s even worse. because, it’s 160 attributes and hundred 60 attributes. Yeah. You, it’s a lot. So it’s probably also filters everything. So that’s what I was talking about, that Magento has his barking a collection where Magento doesn’t promote attributes as a list, but magenta just decides, Hey, I’m just gonna.
you know, I, I just gonna they all have their attributes. I know about, and, I will take this list and I will add attribute one by one instead of a asking AAV model to provide a list of those attributes. So this could have been avoided by using a list of attributes instead of using a single attribute itself.
So it’s completely redundant. And another thing is a MSI. So we see MSI ads around the
Yeah, it’s like 500 that the way square is, right. Soundbite like that. Maybe closer to 600 but. Yeah, this is a white, a lot of them. And
Eric Hileman: [00:39:00] I heard on the database.
Ivan Chepurnyi: [00:39:03] Yeah. It’s hard for them not to waste. Yeah. Okay. 1700. That was great race. It’s a lot. So, okay.
Eric Hileman: [00:39:11] especially if you’re, you’re trying to use a query cash you’re generating, you’re just, you’re going to lock it and you’re going to blow through it.
So many different queries.
Ivan Chepurnyi: [00:39:20] Okay, perfect. Yeah. So
Eric Hileman: [00:39:22] do you, Hey, do you find, do you find yourself disabling or enabling query cache more like which actually you do more often?
Ivan Chepurnyi: [00:39:30] okay. This is arrows. The story is that the mice Creo a team actually work led into disabled. Where your catch by default on every single mage.
My scale installed.
Eric Hileman: [00:39:40] They, they did. It was five, six, I think it was. It was disabled by the file.
Ivan Chepurnyi: [00:39:49] received the community backslash because. A lot of people had to been relying on query cache being enabled by default. So suggests to actually disable it and a lot of people bail. Yeah. Gregory Cash itself is mean to leave closer to his applications into the database server because applications knows much more better, which queries needs to be flushed or into the database server because otherwise server is dumb.
And the other way, Sarah cannot know if a, on this query update, you need to flush a cash for this table because square cash is very simple. It’s cash, a spar table, and every single, if you have any updates, credit as a table is that they will, is going to be flashed. And I think your
Eric Hileman: [00:40:45] lock on it.
Ivan Chepurnyi: [00:40:46] Yeah. and do you lock on it as well?
And Justin Greeson quieter cash actually produces more performance, which makes it worse
Eric Hileman: [00:40:54] figured as the, where’s it is we do, we see some stores it will, they won’t run without it. They’re just hot garbage and you turn it on and they’re okay. And other stores, you turn it on and they lock on it and it gets really bad.
It’s. It seems to just be a case by case. The issue that we have is when a customer starts out, we start out with it off and it’s one of the things we’ll try to get a performance improvement. And if we find that, it’s, it’s hard to explain that to customers cause a lot of times they’ll think, well that’s a server misconfiguration and we say, no, it’s one of those things.
You just need to keep an eye on it and say it helps some customers, it hurts others, and it may be helping you for six months and then suddenly it starts to hurt you. And it needs to say about, it’s not a misconfiguration, it’s just a tuning thing that you need to monitor and
Ivan Chepurnyi: [00:41:45] changes. my square is, was outworkers.
They would run fosters. So it’s, it’s, someone was an interesting things to consider. And also I had some good experience with proxy SQL. Yeah,
Eric Hileman: [00:42:00] we did. On our first version of stratas, we were using a ruler, and we set up proxy as QL in front of all the Magento store. So everybody had proxy SQL on the container and we split the reads and writes.
So the rights were going to the Reuter writer, and then the reads were out over the read replicas, and we even rewrote queries in it. So we would see like, Oh, this is a common. Well, we call a a garbage Clary and we would rewrite them on the fly.
Ivan Chepurnyi: [00:42:29] Yeah. Scale is actually a quiet, powerful duel. But, in general, it’s good for, booting, you know, caching of the queries, because you can select a, we want to figure which Gregory’s, you want the cash and you’re going to run proxy SQL right next to your PHB.
Eric Hileman: [00:42:48] pulls connections to, yeah.
Ivan Chepurnyi: [00:42:50] Yeah. So, so you can, really, do a lot of nice things with proximity. I really like it, but in my opinion, like Brock says, Gail should be the last resort where you really need to scale your, application. in general. You just need to optimize a terrible. We, the soft goods that exists in your applications produces terrible queries.
Thousand something times. Yeah. So here we have another abuser. this is some kind of D cup. they’re good. If you model, could they go to get category a? So good. They already been loaded. One by one, and then there is also some sort of something sorted, sorted. I, I, I see there is some stuff happening. So category top category template.
You see this one has this top Protegrity functionality. probably D cup. It’s just some kind of custom functionality. Yeah. So it’s a view model.
Yeah. Basically he just loads every single category trace taxes. So this is probably some kind of topic. Attackers, a filter or slider. Oh, probably is this fun? So probably this top category is, it’s actually is a category .
Eric Hileman: [00:44:15] Yeah.
Ivan Chepurnyi: [00:44:16] So there’s this thing, loads every single category in order to load the category, image and category name and link.
It just loads every single category separately. So what should have been done here? Jazz grabs those silly stuff. Category IDs, award category collection through repository. Just get the list of categories with needed. That would be, it’s a, you have to access so. No need to, specify every single category separately by, using repository.
You just waste of resources. Yeah. So is this thing produces a lot of that, the waste queries and a very slow, is there any area as a product itself or an odd slow, because. Majority of products I see is that they present things that are catalog are actually just simple products. So it was, there is no configurable products itself.
Nothing fancy going on here. Majority of all of Xero configuration actually is happening here, in alert navigation. So every single, field here. You see is producing quite a lot of unnecessary that the waste gregarious and yeah. So that’s why we see results legs up also is their memory usage is growing very close to a hundred megabytes.
And it means that on some pages that can be him and more because of far all of the categories as their Lord and so on. So. It definitely makes sense. Do one thing, switch to elastic suit and as our thing probably is there also how performance issues was the last six years as well. Because if you have zap many filters, you need to, rewrite the little bit of the last six years of implementation or to reduce number of HTTP calls.
As well, because the last six years has some performance issues too. But the former church, it was emergency. It’s Zen normal out of the box solutions that works much more rather than everything else. Eyelids are available. by the way, I’m actually planning to write a alert navigation steel. is it going to be open source and for everyone?
Is it going to use a Manticore and gonna, do much more foster fields? Or, so we just recently had the, civil communication on Twitter. and I was writing for magenta one. I was writing, before a Sphinx implementation. And I was able to, get to 200 milliseconds, average response time on a quite heavy contemporary page by using Sphinx.
But, I, I was really building Sphinx in a very specific way. So I also utilizing most of the features as it you can use in Sphinx in order to get the FOSS results. That’s back.
Eric Hileman: [00:47:16] Is medical or based on
Ivan Chepurnyi: [00:47:18] Spanx? Yeah. It’s a fork of Sphinx because the Sphinx, yeah. Is a developer promotional who was a original maintainer of Sphinx.
He decided, Hey, I’m not gonna make it any more open source. nothing in that true sense of purpose. So he still provides binary’s for free, but you can see, is it called behind those binaries? So as that’s why I consider it not talking source on him anymore. Right? Because you can to see what’s behind a, is it binary code of seeing search scenes where’s from 3.0?
So pretty 0.0 was the version where he stopped, publicly developing it on, any public, Source control. So it says that’s when Manticore was born and he decided to stop it. Then the, the guy from a Romania, he actually, was also one of the maintainers. So Sphinx project, and he decided to abandon this project and start his own.
gold Manticore and now Manticore is actually used by, as a big companies, is it was using before Sphinx. So before, let’s say Craig Elise was using Sphinx, Airbnb was Susan’s Sphinx. Now you can imagine that, Yeah. Cause they’re using Monte core. So if you take a look at, Manticore a search.
Eric Hileman: [00:48:41] Yeah. Let’s see there. Dev Docker container.
Ivan Chepurnyi: [00:48:46] Yeah. they have a Docker container and basically it’s dropping the replacements for us. Being so is a Sphinx to point to something configurations. We’ll still gonna work with Manticore without any issues.
And yeah, you can see. All as a company. So for instance, Greg least using Manticore, I, I don’t see, actually I was wrong, but yeah, in general, the Sphinx is, less. open Zen Manticore and Manticore is doing lots of things. It’s things was not doing. So it’s really focused to some community and, also abroad sources, much more open documentations and much more better than original OnePlus things.
Eric Hileman: [00:49:31] Now, why, why go with a Sphinx space solutions that are the elastic search, which seems to be where everybody else has headed.
Ivan Chepurnyi: [00:49:37] Yeah. So basically elastic search is building a brand MIS of, everything from HTPA Bay. And, elastic, or, and the Manticore search is kind of a single binary is that you can just, install on your server alongside was your FBM.
You don’t need Java run time. You don’t need to, you know, have a lot of memory on your server in order to run as its solution. So it’s, it’s kind of a, like, so. Let’s put it as this way. So elastic search is like a muscle car, right? So it dries fast.
Yeah. In a straight line. And it’s a lot of fuel. Right? So it could suicides or for sources. Yeah. And a Manticore is more kind of, wary. Optimized a supercar. Is that a. You and probably, drive some electricity. That’s not the gasoline. So it’s, it’s completely different, approach. And, Manticore also, speaks my scalp articles so you can, access, am I scale protocol.
And. The run queries like it would do is my scale by the 12 be against the serous, should not the base, not the gains as a system. Also, and as I, the things that a, I was played neuron, you actually can build a complete a multicore search index in a file system, dark diseases, files and deploy it into another server.
So, if you compare it to, let’s say the last six church and then elastic search, you have a cluster. Is that scales, but in Manticore is there is a different approach. Every single node of Manticore class or can be actually independent completely. So again, just, , index files, and you can start a new instance quite fast.
So basically, if you have, your, , you can actually brings this search server. we’re very close. Does the end user, so let’s say if you want to split up layered navigation, you can, build an API right next to our customer, let’s say in Europe. is that well receives those files. So on a daily basis, let’s say once an hour, we’ll receive a fresh product database and, and ZANU just have a very foster, she ends at three region was out, you know, customer overloading your main servers.
Right for Rosen’s products. So
Eric Hileman: [00:52:17] it’s pretty neat.
Ivan Chepurnyi: [00:52:18] Yeah, and it’s completely independent, so it doesn’t require premium or anything on a server, so you can just, and that could binary and Zissel.
Eric Hileman: [00:52:29] Yeah. Elastic search is massive memory hog, and we see that in so many customers. It just balloons. The multi gigabyte sizes and
Ivan Chepurnyi: [00:52:39] they have very good marketing team.
Yeah. You have to give it to them. You know?
Eric Hileman: [00:52:44] Where we’re not fans of elastic search. We were like three years ago or more, and we’ve done a lot with it and we’ve just like, nah, this isn’t, we, we, we quickly saw like, okay, this has a lot of issues and a lot of things we don’t like about it. And it seems like people are just now catching on to it and really liking it, but I don’t know.
I think it’s kinda ugly
Ivan Chepurnyi: [00:53:10] API. So, And, listen, ABI is not South things that, yeah. So is there a lot of products based on listening? APA, right? You have for solar, you have a plastic search and a lot of those solutions, they are just, yeah, a little bit slow.
maybe for the cloud. It’s good because you know you don’t care much about your resources when you can just buy more. Yeah.
Eric Hileman: [00:53:44] There’s, they’re expensive. Cloud is expensive, man. Make no mistakes. It is expensive.
Ivan Chepurnyi: [00:53:50] Yeah, but I mean like majority would developers consider that a, you can just buy more memory or spin up another node or you know, like, and a lot of people will sit off optimizing as it goes.
If they think about tape, we have got to adjust, make it easier to scale our application by introducing as many as possible garden until the scalable nodes. But yes. You still like low, your costs a lot to result optimizing his actual application. And I think this one everyone will like what I will be able to, because It’s going to have a lot of features that standard magenta doesn’t have and will probably never have, because you would be able to create a rules for even extorting when you export something, the search and Jane, you’ll be able to get an array to the CIL attributes, on the fly. So if you have, they mentioned, so it was a product you would be able to be great ranges of those.
They, they mentioned,
Eric Hileman: [00:54:49] do you have time to build something? Like this? Sounds like it’s, it’s consuming. You have a lot of existing experience and it’s a, you can springboard from that, but still sounds like a big commitment.
Ivan Chepurnyi: [00:55:01] Yeah, it’s a big commitment and don’t expect it’s going to be built beginning next year, so it’s going to be on call, open source projects.
But I think I’m gonna do it publicly. You know, like every single iteration, was every single small step. And basically it’s good I’m being not the waste around the UI. Is it going to be based around the interfaces? And, Service contracts. So basically as a cerebellar, the service itself has going to be a, you probably not containing configuration and data wasted the first stage.
It will just be configured from, BHB objects and to, you’ll be able to, build your configurations by grating objects. At the first glance, but then you will be able to store this as data somewhere as well, and will, gets or speech build groups from somewhere else and you will get the configuration as well.
So it’s not that hard, to integrate. Manticore. Zack, hardest part is actually experts with data. And, for magenta one, when I was building my Sphinx solution from gentle one, I was able to actually write the very, sophisticated exports with the export. yeah. 20 problems with the products was in seven seconds or something like exempt.
Eric Hileman: [00:56:20] That’s a lot. That’s a lot. Yeah.
Ivan Chepurnyi: [00:56:22] Yeah. if
Eric Hileman: [00:56:23] try not to swear because YouTube takes us down.
Ivan Chepurnyi: [00:56:26] Yeah. It’s good to you is they don’t take us down and say, just market as a restricted . Yeah. So, was out with the YouTube account, would not be able to see, some nails and so on. But if you, have an account, and if you turned on the restriction mode as well, you wouldn’t be able to see it as well.
So, yeah. But, it’s not a big problem. I think as soon as we had done sewer on. Yeah. So it was a little bit off topic. let’s go pro with, to a product page. So for this
Eric Hileman: [00:57:08] top category, do you want to go to,
Ivan Chepurnyi: [00:57:11] is, is this a sexually Protegrity was products. So I think that’s a phone. Oh, well that will category.
They have a lot of products here, right? So they have scenarios for the products and navigation is skilling. It alerts navigation is this stop block here. yeah. So,
Eric Hileman: [00:57:30] so elastic suite for them would be a good thing to do.
Ivan Chepurnyi: [00:57:34] Yeah. And There could be some issues with the last issue too, because as far as I remember, is there a, some, a we mutations as well on a number of, filters you have, because they have very, a lot of filters as for standard, magenta, short, sharp, right?
Eric Hileman: [00:57:53] Yeah, that’s a lot.
Ivan Chepurnyi: [00:57:56] Yeah. So, and. Is it problem? Why it’s slow right now? Because Magento, for every single filter, magenta executes a separate query in order to calculate
Eric Hileman: [00:58:09] aggregate how many products are in each one.
Ivan Chepurnyi: [00:58:13] Yeah. And, in order to do so, Magento, as soon as, for instance, just as an example, what does a Brolin was Magento, alert navigation.
So if you select a filter.
And let’s say select a couple of more filters and let’s select the also sort of filter and the forest one as well. Okay. let’s take a look. For instance, neuro restriction.
Eric Hileman: [00:59:00] It’s really nice. I mean, as a, as a customer shopping here, like that’s a really detailed way to find and drill down to what I need, which
Ivan Chepurnyi: [00:59:09] I can’t imagine for a light chest, it’s very important. So, yeah, it’s cool. No, so, let’s profile the page as if they’ve selected.
and do you see right now, is there a snow products right to return from ? is the website.
Eric Hileman: [00:59:34] My wife says she wants a, a latch and a box to keep her snacks in cause I keep stealing her snacks.
Ivan Chepurnyi: [00:59:43] So. So you like to steal snacks at night or,
Eric Hileman: [00:59:47] I don’t know, man. It’s a snack monster. He comes out in the middle of the night and just steal snacks.
I don’t know.
Ivan Chepurnyi: [00:59:53] Okay. So as soon as we don’t have products, you see, we don’t have any more a MSI, Aquarius. All right. Yeah. But we have plenty of other queries and those other queries are, actually related to the temporary tables. So every single searcher field. Oh, they also have a master show by, but seems like it’s not the biggest contributor here.
So, here, every single selected filter results in temporary table. Yeah. So let’s go, Oh, no.
One more, Deborah table. Some are, it should be okay as well. Yeah. Strange head. I don’t see it.
Yeah. We could be guest Magento on every stage. It’s usually creates, at the member table or possible that they just, do it here. Okay. Only in two tables, but then they probably narrow down somehow results. Y’all, let’s see. insert into, so insert into one query.
okay. Not bad. At least here, it’s not that bad. As I said.
Okay, here’s a problem, I think, here, so you see every single query here. First of all, it leads to the debt results. So if you take a look here, say handful, probably configurations that, when you choose a filter, it doesn’t, update other filters. As well. So there’s hypermobility is that, for instance, when I choose something, I will land on the page that will tell me that I don’t have any results.
because you can choose, yeah. So it’s not a standard clear navigation or it’s just navigations that, allows you to choose options. and those counts doesn’t reflect the current, selected filters. Oh.
Eric Hileman: [01:02:37] And we saw shot by in Blackfire, but this isn’t shot by shot. By shot. It’s shot by was amazing use.
They would be causing a lot more performance problems. So it’s like there, but not doing anything.
Ivan Chepurnyi: [01:02:54] Maybe it’s even show by, so let’s see. Oh, it’s not, that one, or is that one? Yeah, here it is. So here is page itself.
Eric Hileman: [01:03:19] maybe I smell Trino modules.
Ivan Chepurnyi: [01:03:24] won’t be a lot of search category. All right. Ken, show bloke for navigation who will be with your flag?
Eric Hileman: [01:03:44] Oh, that’s part of the well pixel thing. Maybe.
Ivan Chepurnyi: [01:03:50] No, it’s, it looks like a standard filter because you know, the square is, looks like, and for every a filter adjust a does a squaring. So. Okay. Unfortunately, Oh, actually, I had the call last week with Blackfire and they showed me a very cool feature that I can find by as a query. If I just copy a query Ian, a call stack here.
I can actually just dial the square in. Okay. Seems like it doesn’t work.
Eric Hileman: [01:04:33] Obviously it didn’t make, because it’s obvious.
Ivan Chepurnyi: [01:04:37] Yeah. As soon as soon as like, yeah. So throw away looks by Zack query. Okay, so a lead’s just is this of fabrication mode, right. And we can do it by using enabled debugging mode. So what it does, it, will preserve every single query.
and then is this mold, I think I can search by query, in the call stack.
Eric Hileman: [01:05:12] That’s pretty cool.
Ivan Chepurnyi: [01:05:14] Yeah.
And major module already has Sphinx available out. There was a box on your servers, right?
Eric Hileman: [01:05:34] Yeah, we do Spanx in elastic search. No, no solar though. don’t see very few, if any, empty stores, using solar.
Ivan Chepurnyi: [01:05:48] Yeah. So solar is where he niche and market for Magento. actually, solar was an enterprise, search module.
Okay. Is this, this kind of crazy
Eric Hileman: [01:06:04] Google damn man. I couldn’t find any results where it like nothing came up.
Ivan Chepurnyi: [01:06:13] I just don’t understand it.
too much. Trace.
Eric Hileman: [01:06:26] Oh, okay.
Ivan Chepurnyi: [01:06:31] Micron just dying on surgery to geek, desktop. Okay. Roll with next time. Okay. Yeah. It just gonna kill my a PC if I’m going to continue. So, okay. Let’s go into some product page, shall we?
Eric Hileman: [01:06:51] Yeah.
Ivan Chepurnyi: [01:06:52] So definitely here it makes sense to a refactor alert navigation and. Oh, no roofing, may be able to get through Instructure.
So here’s the product page, and let’s run a profile. Or here, let’s disable aggregation.
It’s quite fast. So
Eric Hileman: [01:07:22] did you ask them about saving that disable abrogating ag to save all the aggregation preference? So it’s always checked?
Ivan Chepurnyi: [01:07:31] Yeah. yeah. I already did that. Zay a real as ADA to add it as a user preference. So you could share that on the your account level just to disable the way default.
Because for Magento developers, it makes sense because there’s not this fast Deronn then iterations. So let’s see. is there is 500 that the waste credit is probably all of them are MSI. You got the waste squares.
Yeah. And this as well, is this zany dressing, Magento problem you see hundredth and one query. Oh, related to attribute options. I was given back in, 2000. Damn. I already forgot what year it was. I was given an adult, about, making your magenta fly like a rocket or sound full. Exactly. I was giving a talk in a lot of conferences and the one was the main thing I was talking about a lot, is.
Attribute adoptions. So when you are on a product page, you have this local, the additional attributes and the, this, a additional attributes block. Like for instance, here, you see specifications, every single attributes that this a used here a lot. So options separately. So if you have a drop down attribute, Magento will load options for this attribute separately and more interest in thing. This is ethically boots only attributes with cycle adoptions, but the magenta Lords it for every single attribute assigned to this attribute set those product. So product has more attributes was the drop down values.
Magento will Lord for every of to visit. We’ll load the attribute options and what should have been done, on the product page itself is that you don’t load it for, Does that do? You don’t glow that for on attribute level, but you actually do it for attribute options. So if you have an option attribute, you just collect also values to functions was in is this product and then buy this option ATSU just boards or assumptions in one that the way squaring instead of doing it.
For every single attribute or it also options and then magenta as well. Apart from CEDS, if magenta loads those auctions, Magento also iterates through array of options of every single attribute in order to find the right option by ID. And is this a slow operation as well? Because it’s kind of like a brute force search and Sudoku, you know, just doing a quick lookup, you know, to lay the available data by hash map.
So I was already building optimizations for magenta. One of those is kind of preloading and it drops dramatically pages on the customer of app stores where you have a lot of attributes because. Again, we weren’t talking about performance testing, and majority of are gen developers. They never test their solution on more than, let’s say, 10 or 15 attributes being reasonable on the front end.
However, is this customer we already sell, they use 160 attributes, S filterable attributes. Oh, I’ve got equity page. And right now I also see is that they are also using conduit than an 11, attributes on the product page as well. So also said that if you are loading all of the options available in the database,
Eric Hileman: [01:11:04] so is there anything this customer can do to speed that up now.
Ivan Chepurnyi: [01:11:10] yeah, it’s, it’s not something that they can install. so as they need to write a customization is that absorbs, after law advance on collections and on, product Lord and collects is just attribute values for ASOS, attributes. It needs to be shown and Zen by using, as you said, that it with values in jails.
Gonna. A Lord, Is that option labels because basically what this query does, it actually just Lords an option while you’re a Soza labeled text that is available here. So when you would say you have restriction or restriction a size to serious large size, so there’s probably a drop down of sizes. So what magenta does, magenta just floats all attribute options.
Instead of floating one, the ones that are reasonable in the beach. and this is, but I used to see issues with memory usage and it also produces issues with an actual, performance by number of the data. The way square is and the number of the dates as it needs to be processed by BHB.
So, yeah, basically this how this one can be, optimized and, let’s see if we can also, enable all our profile or, functionality in Blackfire and let’s, let’s add something to the cart.
What did happen?
Eric Hileman: [01:12:59] I don’t know. That was weird.
Ivan Chepurnyi: [01:13:02] So I go to the director. Does the main store
okay, so this is, it’s done.
Eric Hileman: [01:13:15] Try to buy now again.
Ivan Chepurnyi: [01:13:24] I think adjust. I’m hardcore that to point to the .
Eric Hileman: [01:13:29] Yeah. Maybe it’s not live yet.
Ivan Chepurnyi: [01:13:34] Yeah. So
Would it be possible? Oh yeah. We forgot to mention that we actually use this time. It’s the first time we review development version of magenta store, not as actual, lively instance, right?
Eric Hileman: [01:13:49] Yeah.
Ivan Chepurnyi: [01:13:51] Oh. And is that a problem in the same instance? You see, I can also see a profile from Zillow instance as well.
Eric Hileman: [01:13:59] Yeah, they probably are.
Ivan Chepurnyi: [01:14:03] And this customer actually has a lot of HTTP requests. Let’s take glucosamine. What’s happening is there,
well, look, labels. Okay.
Eric Hileman: [01:14:17] On an Ajax call, live chat, that we, we had a, we had a customer, yesterday they wrote in my store to slow what’s going on, and then we trace it back and it turns out their it guy installed a live chat on it and it was pulling all the product and carts and data like every.
Every 30 seconds and then pinging it like every second and hundreds of thousands of hits from
Ivan Chepurnyi: [01:14:50] this. It just seems like it’s the same. So we all know is that Magento has already an issue with, customer section lot functionality, right? And discuss the intersection of load functionality is. Sound things at, produces a lot of dope requests.
I think in this, customer case, you will see in new Relic, one was the top URLs related to live shot because this is probably let, let, just for the sake of experiment, let’s go to the homepage.
and let’s enable here profile requests right.
It’s all to regressed. So let me go through a category page.
And bind your more requests. Right? So we already sees that a gossamer section load of course happening, but also get it or get card. get the car results or, yeah, it doesn’t make sense. You know, like it just, I don’t have anything in the cart.
Eric Hileman: [01:16:11] No. It’s real time metrics for the people who are watching a cart and it’ll just crush sites.
Ivan Chepurnyi: [01:16:19] Yeah. So. The, no, what is this? A live chat. A plug in it who wrote this plugin, but all right.
Eric Hileman: [01:16:29] I don’t know. We see a lot of them doing that.
Ivan Chepurnyi: [01:16:32] Yeah. But it’s powered by live chat. So live chat is a live chat. Live chat. Okay.
And this is probably an, an extension available on the market probably as well. So Zay. Did they just decide, Hey, we’re just going to make an ag school. It doesn’t matter. Well, what do you do? So, what is they should have done? actually, put, some, are all in customer section a lot data because.
Got some resection. The Lord already contains gusta motor information. any additional information about the researcher as well, you can obtain through the customer section a lot as well. You don’t need to do any calls separately. You can just request this data from customer section law just rather than them.
And the point
Eric Hileman: [01:17:24] is that the common theme that I keep seeing. Each each week is that all these extensions are operating independently and they don’t really think about what’s going on in the whole as a whole on the site. You know, like, like they’re just like, Oh, I need to get this on. I’m just going to go get this, and not thinking like, Oh, does this already exist?
And I can hook into it. . Well thinking you help with today. Oh wait.
Oh, that’s weird. Yeah, I’ll have to report that.
Ivan Chepurnyi: [01:18:04] So,
and if you scroll down on the page, you already don’t see your drop down. So if you are. Deep into the page, right? If you are deeply into the product
now, but if you move here.
Eric Hileman: [01:18:35] That’s weird. Oh,
Ivan Chepurnyi: [01:18:42] costume select elements. Is that always funny, bro? And because they use some kind of deep Ropper de verb rappers that actually copies the elements, and then just a. Boots, absolute a markup on clique herself like that. So, but this is, I understand is this chatbox is actually some things that is not, of course it doesn’t.
Magenta side, it’s probably some kind of Java script. some Java script on the server part, the server from live chat.
Eric Hileman: [01:19:16] Yes. Yeah. And then, and then, well, it has to, it must be calling me yak to live chat then and sending that cart data back to your master account. so you can see it in your view. The idea being like, I’m the store owner and I’m chatting with this customer and I can see what items are in their car and what pages they’ve visited.
And it’s, I guess it’s sending,
Ivan Chepurnyi: [01:19:41] I think this is the best. Two way to do this. I saw a very nice store, not the magenta store. I think it’s blue order
music. right? Is it it? No wishes for pillar,
I think it is. No. music, Jordan’s or.
Eric Hileman: [01:20:41] Where are you searching for some audio equipment?
Ivan Chepurnyi: [01:20:45] Ah, I had a customer who had does this. A store is a competitor and I’m just trying to define it as the name of the store buzzer was a very nice feature. You actually, on the page or you received, Like here, you can, just receive some kind of a unique quote identifier.
So you can have a string, which you can tell to support representative. Hey, here’s my Colton did the fire or it during the fire fire then buys. This was during identifier. A person can actually look it up. Information about your approaches or how you actually, or help you, was it purchased as well?
You know, jails by, look as you, but. It doesn’t require, let’s say, sending your card information to some sort of partners
Eric Hileman: [01:21:28] every time, whether you care or not, not even using the chat, and it’s getting all that stuff and sending it like, yeah,
Ivan Chepurnyi: [01:21:34] let me go. So I think sort of Scott below for GDPR relations here happening, I need to sends this kind of information to a live chat.
Eric Hileman: [01:21:45] Yeah, that’s interesting.
Ivan Chepurnyi: [01:21:47] Yeah. So
Eric Hileman: [01:21:48] do they have any GDPR thing on their site? And I didn’t see any click here to accept these cookies. Stuff
Ivan Chepurnyi: [01:21:57] I think is a. Wait
Eric Hileman: [01:22:01] cause you, you selected a re okay.
Ivan Chepurnyi: [01:22:03] If I go into your opinion region, do I see, no, I don’t see you in cooking notice. So
probably it’s in each market, but yeah, they need to comply with GDPR anyway because otherwise,
Eric Hileman: [01:22:21] yeah, we’ll send out a town then maybe. We don’t see it or whatever. Okay. yeah. So we, I think we look at all the pages add to cart couldn’t do cause that they must not be set up yet. It’s not live. What were the, what were the big things that we found?
I mean, overall it was pretty, pretty good site I think.
Ivan Chepurnyi: [01:22:46] Yeah. Except integrity page and product page. The product page versus standard program with Magento, as I said, the bottle that abuse. So either with an options, a lot of the other B square is also MSI. Again, as this customer’s brought, we use an MSI because a Zay sell internationally and they probably have different, warehouses and dispatch locations.
So I’ll say, definitely use this functionality, so they actually need to fix on the site itself. they’ll reduce those database queries, as they get started with applying the patches. So we were looking last time. Yeah,
Eric Hileman: [01:23:22] we can link those up again and then
Ivan Chepurnyi: [01:23:25] remove a guard letter. Navigation implementation with something, better like last six suite.
Eric Hileman: [01:23:34] you, you mentioned you had some code laying around for the, a category page, caching attributes. I think we were talking about like the image attributes in particular.
Ivan Chepurnyi: [01:23:44] Oh yeah. I’m gonna put it on GitHub.
Eric Hileman: [01:23:47] Okay,
Ivan Chepurnyi: [01:23:53] But, but this is a very minor, like it’s the last things that you optimize, because a majority of your father’s stuff is actually solid. The beach.
Eric Hileman: [01:24:04] They’ve been working hard on it. I know, from our support conversations.
Ivan Chepurnyi: [01:24:08] Yeah, I understand that they work in higher ed, but, I mean, there’s a problem with lace in a lot of faith that abuse ends is a lot of, had that abuse is not just, you know, fixing your custom code, but it’s, yeah.
Also problem was Augusta gold, as we saw. Is this a dog book at that goodness block, right? And categories that can be fixed and apart from his ads, or is this learned navigation? A filter that needs to be replaced with something more sophisticated, also is they can just. They go could the Manticore or Sphinx and, and also export data Ozer and, just, work with it because Sphinx is very good, was a lot of attributes.
Like you can really send to Sphinx. You can send the cynical query and you can get all of the aggregates. For your car and search results was one query really. So it’s kind of wary us. So I will just show you an example on one of the stores I was working in magenta one error. So it was, or is this efficiently?
stud.com. A store and this website, a Zara using a Sphinx, as is, search a solution for them. So right now, all of this filters, zero coming from Sphinx, and you can, really.
you know, select multiple filters. you can really easily filter about a discount that the route is actually is something, that this is very interesting. You, Suze this one.
Eric Hileman: [01:25:52] Yeah. Yeah. Is it, does it show you how many results you, Oh, yeah, there is two 98.
Ivan Chepurnyi: [01:25:59] Yeah. So, Zay don’t show count, but I haven’t, this extension, I have actually personally strobes at Colin’s as well.
But it only shows you an options that is available to choose.
Eric Hileman: [01:26:10] Oh, so you, so you built this then for am one, and you’re kind of like gonna rebuild it from too.
Ivan Chepurnyi: [01:26:16] Yes. And, basically you see here is they also have information about the available sizes on a product level and all of this thing, all of this speech was, this amount of products is getting ready.
That was in 800 milliseconds. So
Eric Hileman: [01:26:35] pretty
Ivan Chepurnyi: [01:26:35] smooth. Yeah. So this is just Sphinx sense thinks, itself, is something. So choosing, all of those filters and receiving all the possible options for those filters is done for a single thing. Squaring. So Sphinx gives me already back all of this results. And to what is well good about it.
So for instance, Prada shoe, as well, yeah. Syncs as barbering, quick search for exam as well.
Eric Hileman: [01:27:06] It’s pretty cool. Yeah. We’ll have to, on the server end of it, we’ll have to check their Maya’s skill temp table utilization and see maybe we need to increase their temp table memory available memory for that.
Ivan Chepurnyi: [01:27:25] Okay. Other thoughts? Final thoughts? Just. Just just fix your architect group page because cause they’ve already preachers the biggest pain point. I think Ben’s this project
Eric Hileman: [01:27:38] and screw, live chat, any live chat, live chat, suck.
Ivan Chepurnyi: [01:27:43] Z. Don’t suck you Jose John to call Magento, you know?
Eric Hileman: [01:27:48] Yeah. For everything.
Every time, right?
Ivan Chepurnyi: [01:27:51] Yeah. So try to find some kind of smart solution if you, if you need to get some information about the customer. just a supply, some guide of unique identifier and set it to live chat. Don’t, don’t do it with actual bait. That was a customer and your support representative can pick up inventive fire and look it up.
What actually is in the customer session.
Eric Hileman: [01:28:16] Yep. Well, cool man. yeah, so extension providers get at us. you can find us on Twitter. And we’ll link that up and you can email us. and then if anybody has any stories they want reviewed, let us know. shout out at us. Send us an email, we’ll work with you to get a copy over here and go through it.
Yvonne, do you have anything to add before we wrap it
Ivan Chepurnyi: [01:28:36] up? Yeah. You use profilers, you use Blackfire Blackfire school.
Eric Hileman: [01:28:43] Yeah, it is cool. It is cool. We’ve learned a lot. And they’re listening. So you just, you had a call with them recently, right? Like you were talking about the disable aggregation preference and they added a new feature.
Ivan Chepurnyi: [01:28:55] We were talking a lot about actually brewing Blackfire UI, for, specific, application profiling core Magento, because, they were really surprised that actually I use timeline much more than all other features, so,
Eric Hileman: [01:29:10] yeah. Yeah.
Ivan Chepurnyi: [01:29:12] And, Yes, it gels because, with Blackfire, it was timeline and it’s much more easier to support the actual sources of problem comparing to other tools that are available.
Because physical graph, it’s harder to navigate, if you just don’t know where to begin with. So, and then timeline, you have everything in one place and to just easier. The sport, what? The initial performance issue.
Eric Hileman: [01:29:40] Yeah. That’s very cool. I had to use Blackfire in a pass and I really didn’t, I really didn’t get it to be honest.
And then you came back and you were using it and suggested it, and then you started showing me the timeline view and I was like, ah, okay. I get it now. Yeah. That makes a lot of sense. That’s pretty cool.
Ivan Chepurnyi: [01:29:56] Yeah. Good. Comparing to, let’s say the new Relic, new Relic against Bolton issues. Right. The in production, Is that you don’t, let’s say profile yourself, so you can take a look at the traces of a sole transactions. But zip wrong with new Relic, you don’t have, it was the same grand or fewer as you have with Blackfire. So like you can sport, Hey, is this case a, is this Bay becomes slow? So broadly you have to run them Blackwater profile.
What does that specific beige of the slow and find out what actually goes wrong there because a new Relic is good at sharpen your hazer as as this. The other way square is, is it that as good as this much times. But it doesn’t show you render timeline, what the world’s stuck trace and
Eric Hileman: [01:30:46] sequence of events, we’ll call it it.
Ivan Chepurnyi: [01:30:49] then also, for the reasons of performance. new Relic I think doesn’t also captures a wall, a stuck trace. So it only captures the slow calls. And then. A lot of information is lost. Truncated. Yeah. Yeah. We’d be goes, a lot of important formation is actually licensed the detail in the details, like for instance, number of that way squares and whereas those squares has been executed.
Eric Hileman: [01:31:15] Yeah, for sure. Okay, cool. Well, thanks for everybody who made it this far. I will see you next episode.
Ivan Chepurnyi: [01:31:21] Yeah, we wanted to keep it 45 minutes, but we actually kept it a one hour and 40 45 minutes.
Other Articles You Might Like
MAGETITANS UK 2016
We recently hopped on a plane to the historic city of Manchester to rub shoulders with great names in the Magento World at MageTitansMCR in the UK. Technically, our CTO, Marty Pachol hopped on a plane and…
Load In The Loop: Episode 1 – FireplaceParts.com
Load In the Loop: Episode 1 is here! Join Eric Hileman and Ivan Chepurnyi as they identify and fix some performance issues for Fireplaceparts.com. Follow us on Twitter! @loadintheloop Access the git commit here. Sponsorship provided by…
Load In The Loop: Episode 11 – SoundApproach
Load In the Loop: Episode 11 is here! Join Eric Hileman and Ivan Chepurnyi as they identify and fix some performance issues for SoundApproach. Sponsored by blackfire: https://blackfire.io/ Follow us on Twitter! https://twitter.com/loadintheloop Ivan: firstname.lastname@example.org, @IvanChepurnyi Eric:…