The Top 10 Mistakes Made with Magmi

The Top 10 Mistakes Made with Magmi

magmi-logoHowdy,

Magmi is by far the fastest tool for importing & updating products in Magento.

Infact Magmi is superior to Magento’s Dataflow import tool in every single way possible.

However Magmi isn’t the easiest thing to get your head-around especially if you’re new to Magento and in this article we’ll be looking at the common mistakes made when using Magmi.

Video Overview

To help you we’ve also created a video overview to the top 10 mistakes when using Magmi below:

Press play on the video above

Mistake #1 – Not Securing Magmi

responsibilityThis has come first because it’s relating to security.

There are two versions of Magmi available, the developers version that you can download from GitHub and the UnderstandingE version.

If you’re using the UnderstandingE version of Magmi, you can skip this one completely (see the note below on why). However if you’re using the developers version from GitHub then you have a gaping hole in your sites security.

With Magmi comes great power, but that also means great responsibility.

If you don’t lock down your Magmi installation then it’s perfectly possible for someone to delete all your Magento products with one button press and thats just the simple one…. Arbitrary code can be uploaded and unwanted guests can see your Magento database details.

The solutions to this are simple:

  1. Rename the Magmi folder to anything else than “magmi”
  2. Protect the folder using a .htaccess file

We also have a video tutorial on how to do this which you can view here called “Securing Magmi Further

Note: One of first things I did when making Magmi more friendly to the masses with the UnderstandingE version is to create a basic login system for Magmi. I  went a few steps further and created an installation wizard, made the interface a lot nicer to use and also bundled in standard profiles, the common extensions and a few extra tweaks to make it useable out of the box. You can download this version here for free.

Mistake #2 – Making up Headers

Magmi can use the standard Magento Dataflow headers, however as a bare minimum you should be using the headers below.

Before we get into that list, it’s also worth noting that all of these headers need to be in “lowercase”, if you add them in with capitals such as “Configurable_Products”, that just won’t work and you’ll end up with weird errors being returned.

So those standard headers are below and these are included in the example files that are included with the UnderstandingE version of Magmi:

  • sku – The product stock number
  • attribute_set – The attribute set that the product uses
  • type – The type of product, eg “simple” or “configurable”
  • store – Set this as “admin” if you only have one store in Magento
  • configurable_attributes – Used by the configurable attributes plugin, also needs to be used in conjunction with separate headers for the configurable attribute values
  • price – The price
  • qty – The quantity of stock. Also supports + and – values (not many know that!)
  • is_in_stock – Set this to 1 by default, saves problems later with reindexing
  • manage_stock – Set this to 1 by default, saves problems later with reindexing
  • use_config_manage_stock – Set this to 1 by default, saves problems later with reindexing
  • status – The products status. Either 1 = “In Stock” or 0 = “Out of Stock”
  • visibility – Generally you’ll want this set to “Catalog, Search” or “Not Visible Individually”
  • weight – The products weight
  • categories – Used for the sick Magmi categories plugin
  • tax_class_id – Depends on how you have Magento setup. Use “None” if you’re not sure
  • thumbnail – Magmi supports local and remote images. So use either a path relative to media/import or a URL
  • small_image – Same as above
  • image – Same as above
  • media_gallery – A special Magmi column to adding multiple images. Separate more than one image using a semi colon ; character

Frankly speaking anything less than these can cause you problems when creating new products with Magmi.

Mistake #3 – Forgetting that 99.9% of all Errors are Human Related

invalidMagmi has a few bugs, however in my personal experience and from the thousands of posts in the forums:

99.9999999999999999999999% of the time the data that is being loaded incorrect.

If you’re seeing errors being returned, my money is on the product data you’re loading is incorrect for Magmi and/or Magento.

And of course if you’re unsure, go back to the basics and step-by-step course is here: https://understandinge.com/course/importing-products-magento/ and the UnderstandingE community forums are here.

PS. Please include an example CSV of the file you’re struggling with, without this we’d be only guessing at the cause.

Mistake #4 – Trying to use a Magento Dataflow Export File

While Magmi can process the Magento Dataflow export headers, generally its really good practice to use the standard column headers outlined above.

Side Note: Magento Dataflow is the oddest export system I have personally come across. I frankly don’t know how real businesses use it (why on earth would you put categories and images on separate rows? That just makes it ni-on impossible to work with).

Whatever challenges you have with getting a Dataflow export to work with Magmi it will be worth it.

Then use the  file that you create for Magmi as the master file, if you need to make changes, use Magmi. Actually that needs to be Mistake #4A. “Not using Magmi as the primary import tool for Magento”.

Mistake #5 – Configurable Products Come AFTER Simple Products

Magmi configurable items processor settingsThis one caught me out lots of times.

Magmi has a plugin called “Configurable Item processor” (see Magmi documentation here and video tutorial here) that allows you to create simple AND configurable products.

However the “Ace Card” for this plugin is that it can automagically attach the simple products to the configurable product for you too!

If….

You put the configurable product AFTER the simple products and set the plugin to:

  • “auto match simples skus before configurable” = Yes
  • “Perform simples/configurable link” = Yes

And while you’re there, set “Force simples visibility” = “Not Visible Individually” as this is what almost what everyone wants to happen to the simple products so you don’t end up with bloated category or search result pages on your Magento website.

Note: You can learn more about visibilities in this video tutorial which is part of the Magento Fundamentals Course.

Mistake #6 – Not Pressing the Save Profile button After Making Changes

save-profilePerfect timing to point this one out.

After making a change to the current profile you’re working on, such as the options for the Configurable Item processor plugin. If you don’t press the “Save Profile” button your changes aren’t saved. Doh :/

Side Note. An extra tip here is to set up different profiles based upon your requirements.

For example if you have a common instance where you just import “sku” and “price” for price updates for existing products, then create a very minimal profile just for this option. Where as for more complex imports, create an aptly named profile with the plugins you need enabled.

Mistake #7 – Trying to Import Epically Huge Files Using the Web Interface

Magmi can handle imports of squillions of product rows and it’s extremely fast at it too.

However…. Most web hosting for Magento has limits to the execution times. This causes a problem when trying to import large amounts of products.

The answer to this is to use the Command Line Interface (CLI for short).

The Magmi Wiki page looks bleak over here however it really is that simple to use.

Login via SSH and navigate to your Magmi folder and then go down one more folder into “cli”.

From here use this format to run Magmi via the commandline:

php magmi.cli.php [-profile=<yourprofilename>] [-mode=[update|create|xcreate]]

And real-life example is this here:

php magmi.cli.php -profile=Simple -mode=update

Easy eh?

PS. You may need to make the “magmi.cli.php” file “executable” in some web hosting environments.

PPS. Are you wondering which file it’s importing? Thats easy. It’s the last file selection you saved when pressing the “Save Profile” button in the web interface. If you want to force the CSV file name via the Magmi CLI tool then here is an example from the Magmi wiki:

php magmi.cli.php -profile=Simple -mode=update -CSV:filename="/home/myuser/toto.csv"

PPPS. An extra tip here is to use a “screen” session via SSH. Screen is handy little *nix app that allows you to run multiple “screens” within a single session. You can install this with “apt-get install screen” or “yum install screen“.

Do take the time to read the documentation on this via the link posted in the sentence above and the three commands you need to know about are:

  • screen -dMS myscreen – This creates a screen called “myscreen”
  • screen -r myscreen – This attaches you to the screen called “myscreen”
  • Ctrl + A + D – Gets you out of the screen (err detaches you)

Mistake #8 – Err… The Products on My Website are not Showing?

errorsThis catches everyone out.

The thing to remember that Magmi is a database extension, not a Magento extension.

What this simply means is that when you load products into Magento using Magmi, they go straight into the Magento database.

So you’ll then see them in the admin.

However to see them on the Magento frontend, you need to either reindex Magento or save the products individually.

The latter option is tedious, so set your Magento site up to be reindexed each night automatically.

I’m not going to cover the details on how to setup a cron job here to automatically reindex your Magento website because… I’ve already covered this in the UnderstandingE Magento Video tutorials here https://understandinge.com/lesson/draft-using-pathchecker-php-work-magento-cron-job-paths/

Mistake #9 – UTF8 Character Issues?

Blame Microsoft Excel for this.

For example you’re importing UTF8 characters, say for a non English product (German, French etc..) then Excel doesn’t handle the accents.

The simple answer to this is to use Open Office Calc instead and then when saving the file, make sure you set the encoding as UTF8 for the CSV file.

And if you’re suffering from this, then “magic quotes” may help, see this forum post for more detail: https://understandinge.com/forum/all-things-coding/magmi-commas-import-issue/

Mistake #10 – Attribute Identifiers

Magento-Attribute-VariationsThis one is thanks to Dale & the Skype chat on Monday.

With Magmi you can create drop-down and multi-select options on-the-fly with Magmi.

So for example you have an attribute called “config_size” which is a dropdown attribute for size selection on configurable products (this could easily be any other attribute that is not used to create variation options).

When importing products you set the values of this column in the spreadsheet to “Small”, “Medium” & “Large” for 3 differently sized products.

However… let’s now say that you have a new range of T-Shirts and they come in “X-Small” and “X-Large”.

You could go into Magento > Catalog > Attributes > Manage Attributes > Locate the Attribute & hit Edit > And add the options

Or with Magmi, just add in two new product rows and for the new products set the value for “config_size” as being “X-Small” and “X-Large” for the new products and Magmi will add these as new options to Magento.

But the one common mistake is…

Magento & Magmi are case-sensitive when it comes to attribute values.

So “x-small” is not the same as “X-Small”.

For example if you added one product with a value of “config_size” as “x-small” and another product with a value of “X-Small”, in Magento you would then get two dropdown options, basically meaning the same thing to you, but to Magento it would mean two different attribute values because the labels are different because of the case of the text.

Go careful when using Magmi for attribute values in dropdown boxes and my biggest tip here is to standardise what you’re entering for attribute values, especially attributes that you use for creating variation products.

In Summary

magmi-logoMagmi beats Magento Dataflow imports hands down in every possible manner.

However there are some common mistakes that come up frequently with both normal users and developers too.

And for the record, I’ve made all of these and a lot more!

I hope the above article has helped you understand these common mistakes so that you don’t run into them. And if you do, you now know how to handle them appropriately.

If you’re new to Magmi there is a step-by-step course for Magmi in the Magento Tutorials library here.

Matt Ogborne
Co-Founder of UnderstandingE.com

PS. If this article has helped you, help us by sharing this article via any of the social sharing buttons below. And if you have any questions leave a comment below and the forums are always open.

Image sources: here & here

11 replies
  1. Marty Cox
    Marty Cox says:

    When I import products, they do not appear in search or catalog, even though visibility has been set properly. I have to go into each one and resave them without making changes. It says I should be able to reindex and they will appear just fine, but that is not working. I’ve used magmi’s reindexing plugins, I’ve manually reindexed using the magento admin backend and using the magento cli. For each one I reindex everything and nothing I do seems to work.

    The products import just fine. I did a diff on an sql dump before and after resaving the product and it’s adding records to tables like “catalog_category_product_index” and “catalog_category_entity_text”. Any idea what’s going on? I cannot figure this out no matter how many things I try.

    I’m using magmi v0.7.23-git and magento 1.9.1.0

    Reply
  2. Marc
    Marc says:

    0

    down vote

    favorite

    I have a pricelist and have a colomn qty and a colomn is_in_stock.

    My supplier put in colomn is_in_stock a “1”

    I want that when qty is 0 that is_in_stock also is 0.

    I have tryed: new value for is_in_stock: {{ {item.qty}==’0′?’0′:’1′ }}

    then the colomn is total 0.

    can someone help me?

    Reply
  3. Rik Powell
    Rik Powell says:

    Hi, coming from a VOLO background specifically I am comfortable using import/export – shame magmi does not have export function or layout creation?! anyway back on topic:

    Matt, you say “While Magmi can process the Magento Dataflow export headers, generally its really good practice to use the standard column headers outlined above.”

    So this means I can:

    1. export all using dataflow
    2. get desired headers
    3. add desired headers to magmi master file
    4. populate magmi file
    5. import .csv via magmi as usual

    ?

    Specifically other Custom Attributes and Design Layout?

    Thanks in advance.
    Rik

    Reply
  4. jim browning
    jim browning says:

    “99.9999999999999999999999% of the time the data that is being loaded incorrect.”

    Got a few looks in office after bursting out with laughter from reading this. So very true. Great article and site!

    Reply
  5. Brian
    Brian says:

    Matt,

    Thanks for the article… regarding #10… Is there a way to bulk change the case of an attribute? I have tried re-importing the attributes (used for creating configurable products) with both Magmi as well as Magento’s standard importer. The case remains the same.

    Thanks!

    Reply
    • Matthew Ogborne
      Matthew Ogborne says:

      Howdy Brian,

      The attributes are case sensitive, so Attributes and attributes are not the same. The way I would edit them is by updating the attributes table with a find/replace query, failing that reimport them.

      Matt

      Reply
  6. Paul Cartwright
    Paul Cartwright says:

    Hi Matt,
    I am currently suffering from mistake #7 and it has taken a few days to work out, whatever I did I kept having a problems at line 16 but I worked out it is because it is timing out. If I upload the product in smaller file it all works. My problem is I do not understand how to set up/use the ‘cli’ method and from googling it are even more confused. Would you kind enough to show me a step by step guide on how to set this up.

    Once again thank you for your help

    Paul

    Reply
    • Matthew Ogborne
      Matthew Ogborne says:

      Howdy Paul,

      I saw your email earlier. It is a bit of a toughie to get down into Plain English, but…. perfectly do-able.

      It’s in our “Ad-hoc” list of tutorials todo (this post was one of those ad-hoc ones) and we’ll get to it as soon as we can. We need to get the Amazon guides out (they start being released tomorrow), and finish the design course though!

      Matt

      Reply
      • Paul Cartwright
        Paul Cartwright says:

        Thanks Matt

        I look forward to seeing the guide as I have circa 15,000 listings to upload so making 1000 csv file is a long process but do-able on the other hand maintaining that many would be near on impossible.

        Paul

        Reply
      • Paul Cartwright
        Paul Cartwright says:

        Hi Matt

        Just thought I would let you know I have managed to educate myself on the cli method and this was down to great forum post from Jessica which I found from last year. That still didn’t solve my issue but Bob replied to my post and that sorted it.

        Something else learnt and problem solved, Happy days

        Thanks
        Paul

        Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *