Load In The Loop: Episode 16 – Moda
Load In the Loop: Episode 16 is here! Join Eric Hileman and Ivan Chepurnyi as they identify and fix some performance issues for Moda, an online clothing store based in India. What will they find? Can they fix it?
Sponsored by blackfire: https://blackfire.io/
Follow us on Twitter! https://twitter.com/loadintheloop
Ivan: email@example.com, @IvanChepurnyi
Eric: firstname.lastname@example.org, @ericvhileman
Table of Contents
00:09:00 Templatemonster price taken from the product instead of the price index
00:12:30 Templatemonster load of product in a loop to get image (repository)
00:16:00 Images generated on load with png alpha channel for circle instead of CSS/gpack
00:22:00 Webp vs gpack | magepack vs baler
00:33:30 Translation cache not enabled
00:34:00 Dynamically loading styles from database instead of creating a CSS file
00:39:00 Load in the loop with countdown timer and prices
00:42:00 Magento doesn’t case view config by default
00:44:00 More images generated on load with png alpha channel 00:50:00 Stratus efficiency scoring 00:52:00 Search same issues as category page 00:54:00 Ivan brings MySQL search back to m2
Episode 16 Transcription
Ivan Chepurnyi: [00:00:00] So, hi Eric.
Eric Hileman: [00:00:02] Good morning. Good afternoon.
Ivan Chepurnyi: [00:00:05] Yeah. for you it’s morning for me. It’s afternoon already. I already have, the pulled pork being almost ready in my barbecue downstairs. So I put it in this morning and it’s almost completely done.
Eric Hileman: [00:00:18] This is your Sunday ritual barbecue?
Ivan Chepurnyi: [00:00:20] Kind of, but yeah, it’s mostly because we celebrate, right now, like yesterday we had a guest and all was it all the pulled pork that I prepared on Friday is already gone because of guests.
And today we also have, later in the evening we have some guests and this time I prepare a pulled pork from Spanish Iberico pork.
Eric Hileman: [00:00:44] I don’t know what that is. Spanish…?
Ivan Chepurnyi: [00:00:47] A, have you ever tried jamon?
Eric Hileman: [00:00:50] No, I don’t know what that is either.
Ivan Chepurnyi: [00:00:53] Yeah, it’s kind of dry aged, Spanish,pork leg . so a really a dry age it for a very long time, and this is kind of Spanish delicacy.
Eric Hileman: [00:01:03] Dry aged pork.
Ivan Chepurnyi: [00:01:06] Yeah. And they like feed this pork special diet. So the pigs eat only acorns. And they have these black… they have, black, I don’t know how to call it in English. On the end of the leg, you know, you have the feet of the pig… I don’t know how to call it in English,
Eric Hileman: [00:01:24] A hoof?
Ivan Chepurnyi: [00:01:28] But something, you know, like, is the pig steps on the ground with, so you have this one as it is completely black. And, it’s a special breed of pig and it’s very famous in Spain. they made a lot of things from that pig and it has very special taste and I found just a couple of kilos of this pork meat, in one of the Dutch Corker shops.
So I, I bought it yesterday. I was preparing some Dutch, pork, and today I’m preparing some Spanish pork, so…
Eric Hileman: [00:02:09] Apologies to our vegan viewers.
Ivan Chepurnyi: [00:02:11] Yeah. Sorry. And, yeah, that’s what I already did also. You know what? You need to send me some MageMoho t shirt.
Eric Hileman: [00:02:22] Oh yeah. I mean, I’ve had, I hope you wear mediums cause I’ve got all like boxes and boxes of mediums
Ivan Chepurnyi: [00:02:28] You see which shirt I’m wearing?
Eric Hileman: [00:02:30] No, I can’t see. It’s kind of behind the microphone there.
Ivan Chepurnyi: [00:02:33] Oh yeah. Just a moment here.
Eric Hileman: [00:02:42] Oh, no.
Yeah. I will say, I will say Nexcess has some really nice swag. I own a Nexcess hat and I do wear it like a lot, but don’t tell anybody and anybody’s watching this. Don’t tell anybody.
Ivan Chepurnyi: [00:02:57] Yeah, definitely. So no one watching from Nex cess, I should tell to anybody else. Right? I think, cause they will just clip it, you know, like weekly button, put it in
Eric Hileman: [00:03:10] Jordan edit that out.
Jordan edit, edit, edit.
Ivan Chepurnyi: [00:03:17] Don’t forget to put big Nexcess logo behind.
That’s going to be amazing. Yeah. I already see how great this is actually going to be. That’s crazy. Okay. So.
Eric Hileman: [00:03:39] Intro let’s do the intro. Alright. Load in the Loop is sponsored by blackfire.io. I’m Eric Heilman, CEO, and co founder of MageMojo Magento hosting out of New York city. The purpose of this program is to take some of what we’ve learned over the last 10 years of Magento specific hosting and evaluate one of our customer’s store.
We’ll share with you what we find in the hopes that it may improve your Magento skills and the performance of your own Magento stores. And along with me on this magical trip is my cohost Ivan.
Ivan Chepurnyi: [00:04:09] Hey, Eric. Hi there. So I’m, Ivan I’m on base in the Netherlands. I’m doing a lot of performance consultancy and doing a lot of high loaded projects as well as small merchants who struggle with high, hosting costs.
And today we’re going to take a look at some of the customer from MageMojo who are struggling with a performance. They wanted to be in the show for a long time. Right Eric?
Eric Hileman: [00:04:36] This one’s been having some issues for awhile. Typ typical issues, home page catalog, page search. I think you would think we’re going to find a lot.
Ivan Chepurnyi: [00:04:46] I am. Where are we going to check out? What’s going on? Was there, so Eric, tell us a little bit about the customer.
Eric Hileman: [00:04:52] Yeah. So, Moda of the Moda, Dakota dot I N and it looks like they do a lot of, apparel, Yeah, a lot of apparel, I guess, Indian, Indian, specific apparel.
So this is from India, right? So they, most of the sell on Indian Margaret and, they sell, we need to in rupees, as they don’t have probably multi-store.
and do you say is that they have some issues with, performance, right?
Yes. Slow complaints on the home page, category pages and search. No mention a product page is the
Ivan Chepurnyi: [00:05:30] O w we will check all of them. So we’ll see how it goes. So let’s start with, first of all, we are profiling in the home page. I’m going to disable aggregation right away.
And let’s see what’s going to be as a result.
So let’s name it right away. Home page? no, it’s, it’s almost in a Dutch style that she like to put two hours in the words, in order to, make sound long legs. They put a, and it indicates a very long, song sound like then a hug. And so on. And sometimes I just typed too much walls because of that, because I’m getting used to the Dutch language.
Eric Hileman: [00:06:24] How long have you been there
Ivan Chepurnyi: [00:06:25] now? It’s it’s, it’s already for a long, like, I already and I’ve been traveling to the Netherlands all the way back from 2010. So as soon as I started my own company, Like a head of team of developers in Ukraine at some point in time. But as soon as I moved to the Netherlands, yeah, it didn’t work out well.
So the business partner, your decided to go with, TMR in all direction. And I decided to go in other direction because it’s hard to maintain the same level of expectation for the merchants when you are far away from a team. So, Kind of an interesting profile, right? So you see here we have 18 seconds, or was the render time of the homepage.
I think this is kind of in your record holder, on our show because they have a certain 500 doctor inquiries on a homepage,
Eric Hileman: [00:07:23] just perfectly, even 3,500. What are the chances.
Ivan Chepurnyi: [00:07:29] Yeah, this is kind of interesting. So let’s see a timeline.
Oh, I think, cause they just rounded up on the page. So it’s not certified, you know, like Blackfire decided to make it, over by two. Magento usually have off by one errors are usually by 1 cent and Blackfire decide. Yeah, we’re going to be different. We will do it by two. So here we are and is, they definitely have bunch of loading in the loops.
So we can see that here on a homepage, like the health product grid. and there a bunch of the product being probably rendered one by one. Let’s see, let’s see. What’s going on here. let’s see, this one and this name already. It gives me some chills down my spine because I remember the times when I was a very young PHP developer, as there was bunch of, websites or beach BU guy was working on, and there was this famous company called template one, sir, where you can buy some cheap, templates for your BHP nuke websites and say, well, we’re actually setting thin plates for buying show, platforms.
I think. Yeah. So, so is this, this, this seems like, one of their system, like nothing bad about the template, more sort of like they provide a good value for the amount of money you pay for it. The only problems that they don’t have expertize in the platform itself where we say developers, they are focused on so many different platforms for which they developed in plates.
Is that a Z don’t don’t need to invest time in specific facilitation so I can expect, yeah, it says that because of that, we have this kind of a problem here. So if we zoom in here, Will it do it. Yeah. If it’s only here we see as that. Yeah. It’s a future product and there is, some, rendering happening here.
Like get product price, HTML, and just get the product price HTML, does some stuff here. Let’s see what it does. So it tries to grab the final price. Yeah. So is this, this. A configurable product price was over. So this is sexually Magentol problem in this case, at least with product price itself. However, yeah.
Number of queries seven for configurable product is too large. So basically the problem here is that the price is taken from simple products at the wrong time. Instead of being taken from the price index, because in price index, you already have the minimum price of a configurable product. Like you don’t need to walk through all the simple products in order to find what does the minimal price right now and here, what we see right now in this template as it for every single configurable product as a risk.
Hello, the operation happening in order to retrieve, as simple product price and minimum price for a configurable. So this is definitely needs to be fixed. And usually you can just, instead of using the standard rendering was a price. You can just fix the data right away from a product, that is available from the index.
And do you use it instead of using a Z just on the trend of Magento or here, how Weller he are? For instance, I think that another one has other issues as well. And here we are. Yeah. So he have a half. is it our thing called get image? So for instance here, forget MH I see there is already a call to loading of the product in a loop.
So you see here an existing plate interceptor get email, and it loads every single product separately, like frozen, separate that way squarely in order, just to access a single image. That shouldn’t be the case because. I assume go even which is already available in the collection. So as soon as you have a collection of products, you just can specify, Hey, I want to have, let’s say as a main image of a product being available as an attribute and here, instead of adding this attribute to the collection and select a template once Sergio decides, Hey, we’re just gonna load.
Is this. we just going to load this product itself because we don’t know how to retrieve, you know, as, as data for his irregular means. And this is basically laissez health does Zeus amount of queries. So like there was two problems. Is there an issue with the price for three wall? And there is an issue with.
A lot, it was a product in the loop. And if we take a look in the, into the choir released, we can see as at most of the queries are related to loading products separately, like there is by choice by each of them and
Eric Hileman: [00:12:54] that’s for the price. And then the other problem is for the image.
Ivan Chepurnyi: [00:12:58] Yeah. So the price probably contributes around 400.
That the way square is. But the rest of them are related to the, product repositories usage because every single configural product gets loaded through a repository. And then it lowers bunch of files or data that you even don’t need on the product listing because of falls the data loaded. Do you have a lot of database operations?
And this is basically why they have 20 seconds of friends or time. So as a top priority was, I need to do they need just to eliminate this. Lord in the loop of the Getty image Massad, and Zen’s a roll we’ll have already a drop like two, five seconds or something like that. And after that, they can work on a price issue as well.
So it will drop it below one second.
So, you know, like agent time says they can issue 18 times, Foster page, page loading time. And it probably won’t frustrate a visitors who hit on cash to page. And basically this widget brought agree. It’s greeted just used everywhere on home page because they have bunch of blocks, right. Where they show, wireless, products.
And here we have again, Oh, yeah, this is bad.
This is just too bad. if you take a look at, as this one, as this one actually produces a lot of IO as well. if you take a look here, they use this function called image BNG. not only BNG is about four months to serve to your end customers because P and G is not compressed. Right? It’s just our, our, image data is that is very expensive for mobile phone bandwidth.
Moreover, they doing it on a page rendering. And so it means like every time you hit on cached page, you have at least like one, two, three, Four five, six, seven, eight, nine. Them. Yeah. Bunch of brands being Brent images. Let’s see what kind of brand images that they might have here. yeah. Show by brands who see.
So it was those images. Those are PNGs. So let’s inspect an element and let’s see. How big, in, bandwidths, this image has because P and G is even not compressed by broccoli or any other, compressional audience because it’s binary data. So can I open it? Okay. Let’s go to the network tab and let’s just refresh the page and let’s switch to the images.
and let’s go to BNG. So here’s probably our brand team and just right. Yeah. Lager, resized. what is the size of the mange itself? Like hundreds of kilobytes. So this is a lot. This is probably the smallest one because we see here 150. 200. This is too much for this size of the picture. So you see like decisive was a picture is only 200 by 300
And is this one introduces a lot of issues? Like I see probably why definitely promotes or Adidas because you see the email itself. I don’t know if our viewers see, let me check in this one. Does it show overlay on, on the screen? Let me just open up my OBS and let’s just make it always on top. Let me see.
Oh yeah. So you were also see it. so the biggest problem here is. That. Yeah, I understand. What they trying to do is they try to create this alpha channel. A viewer was an email, but they can do the same thing with CSS. They don’t need to do it, by creating IPG image. They just can create a mosque in CSS in order to remove,
Eric Hileman: [00:17:23] the fly.
Ivan Chepurnyi: [00:17:25] Yeah, like is they’re doing it as soon as you have, an image here, like a, as a circle, like Zeta is a source that you mentioned and say, create the circle image. I was an alpha, transparent layer around it, but what do you can do? You can just serve the original image and just, apply seven CSS streaks in order to render it, like all the browsers support as a GSS feature.
So you don’t need to, modifies original image in order to serve it. And Zen, you can use actually G Beck images instead of using PNGs because J pack will probably of this size probably will take like three kilobytes. This is like a hundred times. Lovaza and what they’re serving right now. So this is not the only bad from his perspective as a backend, because you are reciting images on the fly, but also the image itself is very large.
So if someone is on a mobile bandwidth, they expect in India with the scores
Eric Hileman: [00:18:26] travel, that’s terrible.
Ivan Chepurnyi: [00:18:29] Yeah. So as the thing is like, The bandwidth is probably expensive. Like if you want to sell a in country where you know, the bandwidth is expensive. Yeah. Then it’s a problem because also devices are not, you know, is it most, capable to process everything like average consumer in India.
Doesn’t have, you know, top of the line smartphone in order to be able to render all the data, that this sound buys this, webpage. So is there is, there is definitely like things that they need to change. They need to change this once to Japan images and they need to make sure that. They don’t resize those even just on the fly and either they want to get them resized.
They just need to use a standard technology and Magento where you can resize them by URL or some things that
Eric Hileman: [00:19:23] speaking of resizing, what, what is, what does everybody using, major pack
Ivan Chepurnyi: [00:19:28] or. Yeah. So if we don’t come out with email image resizes for images, people use a lot Fastly, right? So yeah, like it’s a quite good, because you can build a URL and you can say, Hey automatically.
So webpage images and pseudo serving your bag. And
Eric Hileman: [00:19:50] let’s say what’s difference between cheap hack and web P.
Ivan Chepurnyi: [00:19:54] A wipey gives additional kilobytes saving. So if browser, send us in these stupid request and tell us, Hey, I support, viewing, with the images and then Fastly delis does his browser. Okay. I will give you the image because it’s going to be much more better from bandwidth.
Like I think it’s around the 20 from 15 to 30% savings, depending on the image. And this is squad two Latino. Like if you consider as a front end performance, a WePay is much more better for a performance of your store because the images faster, is there a smaller, Yeah. So is the biggest problem is compatibility because not all browsers support REPI and our, I would say Firefox, Google Chrome, they support with B.
I think Safaria is now also supporting with B, but, you know, some other browsers might not like. Maybe old internet Explorer users are not able to use Fitbit. I think H is already supporting with B because it’s now based on chromium, right?
Eric Hileman: [00:21:00] The new edge is chromium.
Ivan Chepurnyi: [00:21:01] Yeah. Like a wipey is not that widespread, but you can create rules in your backend.
And instead of G Peggy means you can serve a VP image. So
Eric Hileman: [00:21:13] that’s the kind of thing Fastly I’ll do on the fly.
Ivan Chepurnyi: [00:21:15] Yeah. So that’s what possibly does on the fly. and does that sexually sound things that they planned also to do with this image? Resize our interest?
Eric Hileman: [00:21:25] Yeah. You talked to me about that a little bit.
Ivan Chepurnyi: [00:21:28] Yeah. So each I was going to be an engine X rule where you can just say, Hey, if you have an input, the header for one browser, like it, it notifies you like ISS has this content types. Right. And Zen, when it notifies , you can just create a ride through to add, to change. Should you pack, into web B, if that file exists, like you can create thrive files in engineers and these, that file exists.
You just serves that file instead of JPEG.
Eric Hileman: [00:22:01] And, w it, speaking of like compression and combining, and just optimizations in general, are people moving away from Baylor to major
Ivan Chepurnyi: [00:22:09] pack? Oh, it’s, an interesting question because like, yeah, Baylor is, still, I think I mentioned it in the official report, the results in our software.
So people just don’t adopt it very well because you know, like
Eric Hileman: [00:22:24] I’ve heard mange packs easier to use
Ivan Chepurnyi: [00:22:26] than Baylor. Yeah. And HVAC is much more reserved to use. Like you don’t need to install a lot of things, you know, like he just uses a Phantom browser, frontend JS browser or something like that. I don’t remember exactly.
Or even just use it, just use a screen. Yeah. I think it’s a headless Chrome or cipher like that. So you can sell headless Chrome and then it does a request as a page, as you specify on your local environment. And based on those requests as a pages, it automatically knows which resources to pack. And then it just creates a package for our page type.
So you could, I think you can specify homepage. You can specify category patient, you can specify a product page. And for those three pages, it will create a separate bundle for each of the types. And probably is the general ones that this common between all of those three pages. And, and I think you can easily also, you know, like create, customization to make park because there’s a code is open.
Eric Hileman: [00:23:44] That’s a lot. So how are people, if you don’t have Fastly, how are people optimizing images for G pack or serving alternate web P?
Ivan Chepurnyi: [00:23:52] Yeah. So like, I think there was a juicer, w firms and restaurants.
He also wrote a REPI image. Resizer. how are all those solutions that are fruit, Magento, extension, legs, it resizes a Magento site. It takes a lot of CPU time in order to resize images. So this is the biggest problem with PHP is that resizing images in PHB. First of all, there was a quality of resize operations, not as good.
as for instance, doing it in some wobble level, language where you have access, let’s say to Margiela Japan compression, or you have an access to the, better compression algorithms. So yeah, beach BHP kind of bad at resizing images.
Eric Hileman: [00:24:40] And we saw the IO happening here. So that’s actually happening on every page load.
It’s redoing those instead of just doing them once.
Ivan Chepurnyi: [00:24:47] Yeah. So, so, so it doesn’t do it like once he does it every single time. So like
Eric Hileman: [00:24:52] every single page,
Ivan Chepurnyi: [00:24:55] every single, uncashed. So let’s say even the homepage is not cashed by varnish. Let’s say you’ve Zayed, potentially turn APC off
Eric Hileman: [00:25:07] or someone places, an order.
Like we see so many interactions are operations where cash just gets cleared. We’re always working through that with customers. Like why isn’t my cash dang hot. And we go in and we find this operation or that operation cleared it, like.
Ivan Chepurnyi: [00:25:25] Yeah, it’s, it’s actually just, I was, doing it with, last Friday about the Zetia was number of HTP calls, from the browser.
And also, this is, there was a discussion about the caching. So I was. Giving so many times and opinions that you shouldn’t rely on a cash as a way to optimize your performance. Like cash should be a good saver for you if you want to scale. So let’s say if you have thousands of concurrent Twitters, you don’t have to hit the back end as often.
However, your backend should answer fast enough in order to not restate the customers that hit uncashed page. You know, like if you have cash. Well, what do you look to you? Very high and for most merchants, unfortunately, that’s true because a lot of merchants has some kind of European integration that sends a stock updates or price of dates quite often.
And majority of the, a catalog like 70% of the catalog, usually for those mentions gets invalidated, almost every half an hour. And then you just. Ron, mostly uncashed version of the website. And of course your visitors will be frustrated because they will hit on cached pages more often, like. Well, when you are, is a merchant, like you are just browsing your website and you’re browsing is a common pages you visit.
You never hit the same pages as your users do because your website gets indexed by Google. And when someone searches for some product is actually navigate to the final page, let’s say, whereas this product is available. And is it changed? This is that, as this page will be in cash are very low because.
Usually like the visa does a just browse. The website is a commonplace as they do in categories. Like those pages can be in cash, but most of the people who are visiting the websites, they will go directly into the product page and ZOS product page visitors. They will be frustrated if your product page takes a long time to render or for instance, category page as well.
Like if they hit a diabetic patient, it’s slow to load. Yeah. People will just close the browser when was out, waiting for your page to show up. So you should make sure that you use cash, not as a way to improve your page performance, but as a way to, accept more concurrent visitors, like you shouldn’t rely on it as a way to, Make sure.
Hey, my website is fast, you know, like you cannot improve time to first bite with caching because caching is unreliable.
So yeah, let’s now profile a page. Actually. I want to do something different this time. So. Today, what we’re going to do. We’re actually going to click on profile all the requests in black fire. So we’re going to profile all the, clicks we are doing. So I just want to make sure, you know, like if there is some issue typical happening on the page where we are, we’re in it.
We were going to see it in here in a report. So let’s go to woman, which one you’d like to visit.
Eric Hileman: [00:28:35] Oh,
Ivan Chepurnyi: [00:28:35] I think they were sending some URL. They wanted us to check. Okay. It was, it was general woman category. So I just can click here on top and I probably will go there. Okay, great. So it takes some time I’m going to have some coffee in the meanwhile.
So this is soaring. You see like there’s instead of like, there was a woman, but there is also at least two other requests are happening. so let’s right away. Just go to some product page. You just the first product we, hit here.
And he does a bunch of more shipping requests happening. Right. And more and more of them are coming in. okay. And if we go back and let’s say, I also want to select. Filter because they have a lot of flooded navigation filters, and then I’m also going to select a color.
Okay. So I think that’s enough, data to protest, right? So let’s stop processing and let’s take a look at all the calls that are happening there. So let’s see. this is, the homepage we reviewed and then we see a lot of stuff here. Okay. This is quite a bit you see here is they have the Zim option CSS index.
So it looks like they have a controller that, thanks some CSS data from a file system. You guys chose it as, as a controller. And it also put some water on the database, but mostly they are doing something here. You see file gets CSV, and this is kind of huge, like 10,000 calls through it. This is a lot.
So it’s kind of big CSV file. Let’s see where it gets cold. Oh, it’s translations. So it was a translations. Are getting loaded from my CSV file. So they probably don’t have translation, cache been enabled and they need to work with it. Also, you know, composer is not optimized as well.
Eric Hileman: [00:31:23] How do you even turn off translation cache individually?
Is that an
Ivan Chepurnyi: [00:31:28] individual? I think just cash itself. Like, is there a translation, cash available in Magento and here load module translation. So you see design and load and Zen Mancho stuff and Zinzi have dynamic styles. So first of all, is this a, does this code should not exist in a first place? because.
If you want to create some kind of style editor in the backend, you should store as his style edits as a file and the file system and edits into your deployment procedure. You shouldn’t allow, you know, two dynamic loads or styles for a controller because you are Keating Magento on every single call to the, to the CSS, you know?
Eric Hileman: [00:32:21] Okay. So, now that doesn’t even
Eric Hileman: [00:32:24] Why would you be here? So they, so they have an editor in the back. You, you modify settings in the stores and the, the database for the style. And then it compiles those styles from the database on every page load, instead of just creating like a CSS file that it loads that okay.
Ivan Chepurnyi: [00:32:46] And, and here it is like this file. So. This is actually worried both for front performance because he has, this file is going to be loaded. You know,
it’s not going to be fast, on load operation, you know, as is just, but, and also it’s served with cookies and other stuff. So it’s not the cash. By Warren issues as well. Like
each L’s directly served by, by Magento. It doesn’t make sense. Like this is not the thing you would do. Like. It’s not the SaaS platform. Like I could understand if you would have multi tenant system where you have different URL serving different users. So you want to have a separate styles for different, motions, right?
Who are on your SAS platform. So you are allowed to edit the styles and Edmond panel and so on. However, for agenda, like you are Zoellner as a platform, you are as a merchant, you are as a single installment, you have complete. Access to your file system, doing this kind of dynamic operations doesn’t make sense, because it’s inefficient from point of view of performance as well.
It’s bad for front end performance as well. Like it’s not only bad for several resources. So this is very bad, for your performance and for your servers. Definitely. Also the same thing was new slider, popup legs. They have also separated chippy calls there. Zen, we have a gains M options. And so that it will pop, we have standards, Magento issues with customer section load.
this is what pops all the time. Let’s refresh your page. Okay. So here, we see
some improvements, right? So let’s start with, as a woman could take her itself. Let’s go into this profile. So it was a contemporary page, actually has much more, less database square is an homepage. So the homepage is really terrible and performance. Usually like category pages are the worst and majority of projects, wherever you write her in this case, category page does much better than homepage.
And let’s see, what is the timeline view a year?
so gain Z have some kind of custom product in plate here. At least lay out one.
So here’s happenings a loading was a product, and then here is a trade for products and they probably gained doing some we’ll get product details. Okay.
And here’s some kind of countdown timer. You see, this is bad. So here is some load in a loop. Like they iterate for every single, a simple product or for configurable product. And they have some kind of countdown timers that. I dunno, maybe they have some kind of emulation of the enterprise functionality of catwalk events where you can, give a limited offer for a price, or maybe they are using standard Magento functionality or special prices.
It’s kind of interesting. Like, does it count on time or what? This is completely wrong way to do it? Like. They’re doing a, the w separation on every single product. They should probably add an observer for after collection Lord. And based on that load information about the product special prices, what is the earliest date when special offer ends byproduct 80, and just provides this data for this countdown timer, you know, like this one.
Actually is very inefficient. awesome. Are the same price in issue of Magento, like get product price again, the same as we’ve seen on homepage, where every single product of configurable gets it’s rated.
Yeah, final price. And then there’s this countdown timer. So this is basically the performance issue is mostly a countdown timer and prices. So this is contributes to it. All of this, the other way square is, and this is definitely going to be optimize. Let’s see what’s going on here in literal navigation rendering.
some kind of things happening here. Kind of get swatch data Chis. So, let’s see what was going on here as a switch data CPU time is quite large. Unfortunately, I don’t see as a code of this method, but I think like, There is some kind of inefficient PHB algorithm is that, you know, like the rates within the loop rate salute one more time or sample X adds up steaks.
A lot of CPU cycles usually like Seoul with, by creating some kind of hash map when you load the data from the database. And then you just look up your hash maps. That is very fast process. Like you don’t have to trade over and over at the same, it will all the time. Zen here. We have a swatch path and here we have some kind of Alice.
So here is our view configuration being sedated, actually a couple of times. And because of that, they have multiple, operations happening. So this is a standard to Magento core problems. This piece of a code itself that is possible to fix by, using, by using some catching around the view config file Magento doesn’t cache view config by default.
I don’t know why, Zen here we have price range. this is kind of looks as well bad because there’s a lot of the other way square is,
and this is not looking good because like, Good current prize, get month’s prize. Get Quinn prize, good market price. Like this is too much without the base operation, just to retrieve the minimum price in enterprise slider and maximum price and the price slider. Like there should be maximum one database queries.
This is too much queries and they’re quite expensive. Like 20 millisecond for database query just to have minimum and maximum price is too much. So seamless, like template monsters. They also tried to implement onload navigation because as you see, like, has this price slider and most probably like they just don’t, optimize and say code their code in order to create those price buckets.
Go see if we go here. you know, like me clear all the, current glue applied filters and here, right on the price, like what’s they need to have, they need to only know what is the maximum price and what does it mean when, when, what does the maximum price they don’t need to know? so many times, so it’s enough to have a single query for that.
And here we see, is it, where is it? Whereas the profile I think here. Yeah. And here they’re doing it a lot of times and they’re doing a lot of pinks and as well, size sidebar as well show by Brent. So every single category page, again, they say, and they made sure I was a brand itself because April would have a brand somewhere here as those small images.
You see? So again, those brand images. Are very ineffective from point of view of performance. Like Z just take all the set of resources. I don’t know if this customer has a lot of concurrent tweezers, but probably if they have a or a spike that will they cook them down, we go slide is this amount of eye operations is very bad.
Eric Hileman: [00:41:28] Make stress cry.
Ivan Chepurnyi: [00:41:30] Yeah. Okay. And yeah, we can see, for instance, when I are narrowed down the results on this woman page, when I applied some fields that I see sees the number of pair is dropped because number of products being shown got reduced to only one product, right? So a number of other basic operations yes.
Reduced dramatically. And that’s what they can expect when they actually fix, the load in the loops, in their pages. Like the regular page would render as this amount of time. And then you see. There’s the biggest problem after is that this actually resizing the image. It’s not rendered and over the product.
Eric Hileman: [00:42:11] lot of this isn’t there seem though. So how do they, like, what are the big wins for them to optimize this? So they have to find a new theme and rebuild things like, is there anything they can do to,
Ivan Chepurnyi: [00:42:20] Zaycon fix, performance issues in Zim now? Like they can invest time either to rebuild everything from scratch or actually just optimize performance of the current Zim parcel team.
Yeah. Like I understand like, yeah. but if they’re business relicensing, you know, like on how it looks and so on, like they already invested some time into it. It makes sense to fix is in like an IDE I don’t see like performance issues that are impossible to fix with because yeah, just moving resizing operation out of, you know, housing doesn’t sound that complex.
Bryce, fix us. Those are bars. It was the corporate ones that they think can just do couple of plugins in order to recuse the data from pricing Dixon was ruins the data from a as a final price. So by trading for simple products, and then also, this countdown, components that. And image a URL component of the product list.
Is it going to be fixed by just not loading product in a loop? So
Eric Hileman: [00:43:26] some data to get their developers to go in and look at the parts of the team that we talked about. So at least they know what areas to focus on and the theme to optimize.
Ivan Chepurnyi: [00:43:36] So let’s take a look at the last one was a, wor checking as an was, they were saying was a performance issue.
So with me, cope is just a URL. And let me start actually profile requests. So Blackfire will have comedically roll file. Every single URL I visit from that one. So let’s see.
So this has got to look search page and let’s stop a search and it will take some time, for Blackfire to load the stuff. So you see, like on every single page, like even if it’s caused by varnish, there still will be a couple of HTP calls to load CSS styles. So. He was, they have a spike in visitors is able to have an issue with that because Zabel gets FBM workers, depletion, and.
As less fem workers you have Zoe boot is that someone will be able to add something to the shopping cart and actually proceed with something. So basically you can run an ad campaign, but your web shop is unable to handle it. Because it’s busy showing CSS because it’s busy serving CSS styles to Azara visitors, you know,
Eric Hileman: [00:45:00] fortunately for them they’re here and we’ll scale out PHP, FPM, but yeah, anywhere else that’d be a problem.
It is pretty inefficient for us to run this store.
Ivan Chepurnyi: [00:45:12] Yeah. But even like for you, you can do it only to some extent, you know, like Zara and starter plan. Right. They want to receive, you know,
Eric Hileman: [00:45:21] there’s no limit on PHP, FPM on any of the plans. They can scale up at the database. You’ll hit the limit of what your database can do
Ivan Chepurnyi: [00:45:28] so they can abuse your system.
Eric Hileman: [00:45:31] Yeah. That’s what that’s, what’s happening. We are, we’re starting to score customers by efficiency and, targeting to help them out. And the more, so this is a whole internal discussion. I like here, we have this, This car insurance company has come out with these devices that you plug into your OBD, two port on your vehicle, and you drive with it for six months.
And if you’re a safe driver, they give you a discount. So we’ve started scoring customers and we’re looking at giving the efficient ones, the good drivers discounts on their hosting. So. But then people like this, we have to figure out like, okay, if you’re really like outside of the, you know, semi standard deviations from what’s normal efficiency
Ivan Chepurnyi: [00:46:15] to address that.
Like just a general, like everything about 200, that the way square is, is bad. So a hundred that the blue square is just kind of pick out of point after which, you know, like you cannot be available for a discount, no matter what kind of a page like 200 queries is too much.
Eric Hileman: [00:46:36] We were looking at sessions. So we have sessions and we have CPU time.
So we actually have all the CPU seconds used from all the services by service. And we can use the CPU time that the customer’s using and memory as well to say like, okay, this is your efficiency score. So people who get no sessions with less CPU are more efficient
Ivan Chepurnyi: [00:46:56] essentially. So we can actually have some kind of a calculation.
On a number of visitors correlated to the CPU time used, right? So the less CPU time you use per visitor. The more efficient you are, this is kind of where you need graph, you know? Like, and it’s going to be interesting to check out because yeah, this is great. They’d be, yeah. Yeah.
Eric Hileman: [00:47:23] I’m sure customers like to see how they compare to others.
So we got to break it down by and. I’m curious to see what happens with the different versions of Magento too, as well. Like we were testing 2.4 and that seems, seems to be running a lot better. So we’re hopeful that it has more efficiencies in it.
Ivan Chepurnyi: [00:47:43] Yeah, I hope so as well.
Eric Hileman: [00:47:47] You don’t sound very optimistic.
Let’s talk about that at the end, actually what you’re doing with search and everything, but I want to talk about that at the end. Let’s go get through the,
Ivan Chepurnyi: [00:47:58] yeah, so we page, and this is our catalog search page. And as I see, like it’s completely the same, performance issues as we have in calligraphy.
So they basically have this brand sidebar. Is that and zinc halves is slowed in our loops for is the same template. So it’s product, we slay out one. So it’s the same team plays. It was used from category page. So basically Z shoes are the same in both places like the degludec page and the search result page.
they’re both have the same performance bottlenecks, as I already mentioned before. So let’s, those, the last theme, sound things that the customer didn’t complain about, but we probably have to check it. We have to check, add to cart, right? So let’s add to cart and product and let’s see how it goes.
Your tab just crashed. Okay. That’s nice. Is this one I didn’t expect?
Eric Hileman: [00:49:06] I haven’t seen that before
Ivan Chepurnyi: [00:49:09] and yeah. It’s sometimes happens, you know, like maybe. There was some kind of billboard sound fair. Like that was okay. So because it’s configurable product, they have to choose bunch of options. Like they have to choose a price before I can add the product to the card.
So with me profile all requests now, and let me, add this product to cars. It’s nice. be a medium sized dress. Let’s see how it goes. Okay. Edit. Drawstring led address to your shopping cart. Great aisle. And let’s right away. Proceed to checkout. no, no, I don’t. I don’t want to create an account. Sorry.
We, we want profiles and check checkout because you know, I didn’t want to create an account in order to do it. So let’s see. We have checkout cart ad.
Great. And here with how I’m not the match of foremost issues. Right. So we have some starter thing. Agenda has, I don’t see, you know, anything I’ll afford nunnery? yeah, from the time, because 701 seconds. It’s a good one. Yeah. So it’s standard Magento on like they had the product Z collects dolls, you know, like it’s not from gender profile.
Yeah. Maybe it’s not the best performance, you know, all fed into cart, but this is standard Magento and, you know, like comparing to other performance issues. They have, they don’t have issues with that to cart because Jess rendering products takes a lot of time. Okay. So. With this one. I’m happy. So, yeah.
Let’s talk about the stuff, about blurred navigation. Yeah.
Eric Hileman: [00:50:59] Yeah. someone asked me about, like searches, my SQL searches bad. Why, why do you want to bring my SQL search back? I guess start
Ivan Chepurnyi: [00:51:07] there. Oh yeah. So basically, is the biggest issue here? Is that. You know, like it’s not that my Souk role is bad.
It’s search my sickle full text search in giant squad, Guten 5.7 and 5.6. my SQL versions like Z, before like full tech search of my SQL was bad because it were, first of all was only my Assam base table. And, you know, like mixing my assignment in two B is not good in one database. Let me guess, you know, like you cannot use efficiently your, my SQL server, however, as they brought a full tech search into, you know, DB, and now basically you, you can do a lot of basic search without a lot of things as the only things that for my SQL in order to have your search working good, you need to properly.
Way to your fields you’re using in search. And do you need to properly prepare the data? I don’t know if you saw my tweet with the proof of concept of search suggestions, like where I was able to actually, you know, just run, On the 1 million products. I was grading some serious suggestion data, like a generated 1 million products for search edition.
And it was some for like a 300,000 to have a suggestion strings itself. Yeah. For a week, my SQL was able to return me a result within 70 milliseconds of the search suggestion with after corrected a query. So I, I use some faker data and it was great in some, concrete product, chicken, type of products.
So. It was an interesting thing, you know, like, because I’ve built was building for Magento one, I was building search a solution based on things. And what I learned from building and working with full text search and giant, like, cause there was a lot of things you have to prepare your data first before you actually have it done properly.
Like search possession is easy to create. If you know what. So how to actually split your keywords into, to grams. So like you can have a, let me actually show you maybe. so yeah, I, I I’m blind text editor or something like that. I’ve
Eric Hileman: [00:53:30] read about tri grants before, but there I’m completely blanking on them and Sunday morning,
Ivan Chepurnyi: [00:53:37] so can I actually make it larger?
Okay, great. So, let’s say you have a war chicken, right? And you have a work product, right? So chicken product
Eric Hileman: [00:53:54] chicken now.
Ivan Chepurnyi: [00:53:55] Got it. So try grams. It’s a way to, split warts into, Dry three, like three character, segments. So you create like cheat, you create a C key E you create. N you create B air, you create UK pry you create the, you usually also create more permanent permutations like Shi, and then you can create a, he Keke.
You can create a, let’s say. okay.
Eric Hileman: [00:54:38] So,
Ivan Chepurnyi: [00:54:39] and, and so on. So this is a scold trigrams like you have a search keyword and do you like grade three with rewards from it? And then you have the same kind of indexation happening when you create a search index. So create a field with strike grams, and then you run a full pic search.
And as more trigrams matching your search result as more relevant to your search result is
Eric Hileman: [00:55:05] so that’s, that’s always been a thing it’s not so much search technology. It’s the data and the configuration of the search. And what you’re writing does a lot of this automatically, or is there still a lot of customer?
Ivan Chepurnyi: [00:55:16] Yes. Okay. Basically what I’m planning to do, first of all, is going to be. It very efficient, layered navigation. And my sequel is it’s not as the case was Magento two because Magento two creates this batch of temporary tables that are completely unneeded. They’re like you just skills, your bicycle server.
It’s much more better to use a common table expressions, in order to, you know, like credos those to results sets. and. Yeah, that’s basically what is that? lyric navigation is going to be about apart from Seth is going to have bunch of other features that are not available even in started Magentol red navigation.
Like it will have what’s the ability to choose multiple filters. Like somethings, if people use surpass the extensions for is going to be just an open source extension, just wasted slurred navigation at a place of . And do you have bunch of features? So is there.
Eric Hileman: [00:56:10] Are you going to replace smile, Alaska sweet elastics.
Ivan Chepurnyi: [00:56:14] if you are not using Elasticsearch, you will be able to use this one. So you don’t need to have another compute instance. We see a lot of memory in order to support a memory code called elastic search, you know, because it’s a memory hog. Yeah.
Eric Hileman: [00:56:34] Okay. So if somebody could run elastic search or not, or if they don’t, if that’s not an issue, would they be better off using what you’re writing or still using elastic
Ivan Chepurnyi: [00:56:45] suite?
it depends because, It will be possible. Like I build an extensions that you will be able to write and integration with any kind of search giant answer. Like it will provide by default, it will provide my SQL based storage. but, the design, is they can account, like you will be able to write the fast index or like has the biggest issues I have right now is the, let’s see what, like it’s good solution right now in the market.
And. As a quick solution. I usually point my customers to it because you know, like it’s already developed, it’s already there and it’s minimum effort, but what I’m planning to do, I’m planning to build. his solution is it will be super fast to index because I was building before is things based, alert, navigation for Magento on.
And I was there as there was numbers. Like you will be able to, you will be able to export a 200,000 skews in seven seconds to search in giant. So right now, if you want to do it through elastic storage to be API, it will probably take like an hour or something like that. So you cannot really run fast indexation with lots of suits.
So it reduces your time to market. When you want to run a new campaign, let’s say, decide, Hey, I want to quickly give a discount on half of my catalog because you know, like there is this day and I want to do it right now. You kind of do it right now with elastics, you, without planning in advance, like you, you, you cannot have this quick market reaction, or for on, for instance, like, I’ve seen so many customers, like changing the price.
Like making a type way in a price and given a ridiculous discount in the catalog. And then it takes a lot of time actually to re-index it back. There’s a proper price and they actually lose money while, you know, as the old data is there. And this is kind of, things that. I’m a fan of having something fast and something easy to use.
And based on this extension, I’ll be able to maybe beautiful. So something proprietary for something for Manticore search or something like that was Ross then. So, so, but this one is the point of another discussion, but in general, like open source project is going to be, interesting because I’m gonna great.
A lot of. The staff that they already have in Sphinx extension for Magento one is that they was in song for merchants. And I’m going to create a possibility also to have CEO based a filter of URLs. Is there going to be a fast, and not, duplicating millions of records in the database for your rights.
Like majority of the stuff happening, I was show buys and ads are an extension salads there. He’s going to have also a real show, a category data. like I already was shoving, I think multiple times. And example of, is this website. so this is one of my, Magento one, customers, and they have these designer category.
And here, for instance, if you choose a brand, Versace, You see, as the Integra data changes, like they have a Bonner for Russia, Saatchi, you know, like even, if you just keep Zeus URL, like right now it’s sort of loaded through a box, but if you hit this URL directly in a browser, you know, even title of his page, he gets changed to.
I would have Staci allEquip. So it was a basic idea. Like you can specify based on the filter combination, like you can have product type, a gender brand gall or whatever. You can create a customized, category data because Zane giant will check, Hey, we have this, this filters. Do we have some replacement category text, which we can use, to display instead of standard category data.
And then you have unique CEO page that you can just create the promotional campaign in Google ad words. And you can say, Hey, hot summer sales was out, creating a new category just by specifying filters and, grading some proper sort order. Or was that her and just shooting it, into your AdWords campaign and selling products, you know, like.
You just save a lot of time and management of the data. So this is kind of a features that I want to bring to community in open source version. Like I want to half, you know, some kind of an advantage for merchants to use a. You know, Magento, basically, I want to have SMB being able to use Magento was out, you know, paying large amount of money for the hosting.
hopefully it will pay off in, you know, in future for me by, me being hired by those merchants. You know, for us our stuff, but that’s all Azar things. So that’s basically what, I’m busy right now. And I’ve already had the couple of live streams where I was actually doing design, with other people was other developers and people was giving me some good feedback.
They really liked the idea. Oh, all those filters and also an ADL, or actually having filters being decoupled from Magento itself. By right now in Magento, you have filters that are completely coupled through your attributes. However, sometimes, you have in a store, let’s say this customer, right? So if we take a look at their category, you go to woman category and you just go into colors, right?
So, is there a lot of colors and I highly doubt it is good for an end user when you have so many choices, right? Usually, you’d like to reduce number of colors, but you don’t want to specify is they associate at a color group, you know, where each product, What I was doing before, like I was able to create visual filters.
Like you create a new filter where you have, well use, let’s say color. let’s say red color would be a group of Rose, dark red, lie trapped. Well, you know, like all those color variations, but it will be only one color group that you will be able to, filter out in alert navigation. And so on and the same thing, like, when’s that on fishing East, the store, like they also have a sound reshow filters as well, like, as to what,
how many days colleagues. Right. I didn’t know. so. Eight or for instance, you see this discount filter and do see on each product, like you have an actual number, like minus 41, my minus 62. And so on. So as these discount filter is works in such a way, is that you just specify ranges of values and then each will just specify products within this range.
Like there is no special, a drop down attribute with discount on the product level. It’s just automatically going to rate it. The filter based on the data you have in your catalog.
And this saves a lot of, data in your database because you don’t have to create a new attributes for products. You don’t have to do a lot of stuff. You can just let’s say specific. Hey, I want to sell, let’s say furniture and I want to create a filter for the furniture. Let’s say between this size in, feats or centimeters between that size and Z size.
I want to create trenches, but I don’t want to. Specify all those ranges per product, because right now you have to do it in Magento. Like you have to create a new attribute, a drop down the attribute, and then you just do it like that. However, I just want to have a single let’s say integer or decimal attribute based on the week.
I can just generate this data. Like you just tell me the rules. Hey, this dropped down. Well, he was from
and the merchant is just. Focusing on marketing campaigns instead of focusing on how to enter the data core products.
Sounds cool. Yeah. So.
Eric Hileman: [01:05:31] When do you think it be ready here? What’s your next step? You’re going to put it. Will you, did you put to the PSC up on get hub? He puts out
Ivan Chepurnyi: [01:05:40] is a POC was for search suggestion, like based on trigrams when I was able to, you know, to have to create the quick search queries or something like that.
Some things that is possible to do, like you can even get new rate the Zillow squares, like there will be getting a raise where we will. We’ll be able to choose like a, what kind of search positions you want to have, like breadth of the product, product type, we’ll say size scholar, whatever, you know, like you would be able to create multiple relations and it will just going rate off the suggestions based on it.
We’re going to read three grams and then person will typing instead of. The default one, when you only see already used search positions, it will suggest you actually search queries based on the data you have in your catalog, instead of relying on, you know, bait does that, someone searched before. So it’s much more better user experience because it’s very right away.
We’ll get some hints, you know, like how they can build their queries. Yeah. but yeah, right now it’s all open legs. There is good hap repository for letter navigation itself. Like design is only containing right now. Interfaces like the idea how’s it , right now also I work on as Ukrainian, as my SQL batch library for Magento as well.
Once it will allow me to build efficient indexers for letter navigation, as well as final report. My import export tool as if they have, in one of my private repositories.
Eric Hileman: [01:07:14] He had a lot of stuff in this dash.
Ivan Chepurnyi: [01:07:17] Yeah. I have a lot of stuff. Is that. Unfortunately, I can’t release it as open source thing as well.
Like I’ve, I’ve built a lot of custom solutions, or as a years, but yeah, I just need to pour them over. Like I cannot reuse the same code. So as, as that’s why I have to build a lot of stuff from scratch. It says that stuff. but in general, yeah, there’s a lot of projects, that I’m working on right now.
And finally, like I have some customers who are interested in those projects, so Barsha is a support me on getting them done, you know? And so having Magento deciding to teach, my SQL search, it’s kind of. A big, it’d be a driver for me actually to get this slurry navigation down as soon as possible.
Like, yeah. It’s probably take a couple of months to develop it, you know, because I don’t have full time. I will do it too to build it. You know, like it itself, it’s not that tough to build as a Rome is usually time, you know, and I have to feed to my family. So I have to take customers. I can just work for free for open source projects.
So I dedicate some time I get some sponsorship from my current merchants who are ready to wait for it, you know? but I think in general, yeah, this is. this is South things that I think going to change a lot, market in Magento, itselves that a lot of people will be able to sell more with less, Holston resources being spent because dosage will be much more better marketing tools and the folder navigation.
It’s going to be much more better, you know, Indexation speeds. So time to market
Eric Hileman: [01:09:11] and other assault like that, they make money by selling you more resources will like it. I mean the more optimized your store is,
Ivan Chepurnyi: [01:09:20] the cheaper it is for us to run gamut. I hope it’s going to be a good competitor for Magento standard search and navigation as well.
Because I think it’s going to have much more, but the features and people will just switch to standard SQL because they don’t need to install an additional software on their infrastructure. Like it will be just working with standard Magento, stellar from gentle the day already. How, and when they decide to upgrade to 2.4 from 2.3 brains, they’re already will have something to immigrate.
Too, you know, like they won’t be forced to change their infrastructure, change these setups and so on because I think decision enough, Magento to leach my SQL by slurred navigation. he is quite bad for, SMB merchants
Eric Hileman: [01:10:18] and they’re moving that all into elastic search.
Ivan Chepurnyi: [01:10:22] Yeah. So this, this is, a lot of things I think is, are moving into PWA. Direction like, and they don’t want to support multiple things at the same time. And, everyone knows that my SQL implementation or the default Magento navigation just terrible and they know it themselves as well.
So that’s why they trying to build this thing. and they just want to teach it, you know, like they don’t want to fix it. They just want to switch on something to focus on because they have developers like supporting to. Systems to have identical functionalities, and perform well is always hard, because the internal design, unfortunately of the lateral navigation is not built in such a ways that you can easily, you know, span out multiple implementations.
Eric Hileman: [01:11:14] Yeah, our testing of 2.4 showed pretty good performance increases on it. Now that’s just, that’s just demo data. We haven’t seen any real customers obviously with it yet, but
Ivan Chepurnyi: [01:11:25] yeah, it’s just great. A lot of configurable products.
Eric Hileman: [01:11:30] That’s yeah. I had a couple of extensions and then it crashes again. Okay. well let’s, let’s wrap it up.
So thank you to Blackfire for sponsoring us, like subscribe, follow, and we are, load in the loop on Twitter and L I T L at dot com and I’m, we’ll link up our personal, contact information in the show notes. Any, any parting words before we head out Ivan?
Ivan Chepurnyi: [01:11:55] no, I think it’s good. I think we, we covered a lot of topics already today.
Thank everyone who tuned in and TEALS the next show.
Other Articles You Might Like
MHM Dedicated Plans Ending – Increased Performance w/ Stratus Cloud
We’ve made the difficult decision to shut down our dedicated server plans on March 1, 2019. As you may know, we have not offered dedicated server plans since last year. While we understand the impact this will…
Load In The Loop: Episode 9 – Black Swallow
Load In the Loop: Episode 9 is here! Join Eric Hileman and Ivan Chepurnyi as they identify and fix some performance issues for Black Swallow and struggle to understand the world of makeup. Sponsored by blackfire: https://blackfire.io/…