Yes, of course, this really may be the best compact system camera you can buy these days. That is, unless you agree with Ken Rockwell and want warmer colours and a better built-in flash. To me, the former may not be a show-stopper, but I would probably miss the perfection of the built-in flash of my X100.
But I digress. What I want to tell you about is how pretty much the whole lot of these new compact system, interchangeable lens, cameras are mostly just plain wrong.
You see, regardless of undoubted image quality, versatility, and probably a few dozen other great features they all have, one thing where, for me, they fail is balance. Simple, honest, balance of both looks and, well, balance.
I mean, of course, with a small, probably wide angle prime, you still get something that is pleasing to hold, and doesn't feel awkward and front heavy. But go for a nice big portrait lens, or even just the kit lens these things tend to come with - especially the kit lens which, being a zoom lens, extends, too - you get something that just can't feel good to hold and balance, not to mention that it also looks a bit silly (well, it does to me, anyway).
You may now argue that the overall size and weight, especially compared to a DSLR, outweigh my objections. But do they really?
If you're going to mostly use your X-E1 with just a short wide prime then you could have saved yourself almost a grand if you went for the X100 instead. And with the X100 you also get a better flash, too.
If you plan to use mostly the normal zoom kit lens, then the overall size, due to the length of the lens itself borders on carrying a DSLR with a similar lens. While this may not be true inch for inch, it would certainly feel the same. I don't yet see very long zooms for compact system cameras, but these would be even bulkier.
And if you plan to use your expensive compact system for portrait work, then, first of all, you get to attach a lens that is similar in size to the kit zoom when zoomed all the way out. Unless you plan on doing candid street portraits, for which a DSLR may be too conspicuous and/or noisy, then again you may be better off - at least in terms of costs - with a DSLR and a portrait lens. For example, it seems that the excellent Nikon D7000, with a somewhat older, but still excellent, 85mm f1.8 prime, is again almost a grand cheaper than an X-E1 and its 60mm prime (which is at least a stop slower, too).
Finally, if you plan to haul a set of lenses for your compact and change them as you need them, then almost all smallness and convenience factor gets rubbed out by the fact you still need a large, well padded bag to carry the lot. You might as well take your DSLR instead
Objection! I don't own a DSLR. A compact system camera is all I need.
Overruled! Look at the first two or three reasons I gave. The balance is just wrong. It would feel awkward in your hand, and you may find the stability compromised, too. Of course, you may not mind, or you may prefer a compact to avoid shutter noise, but that's beside the point here. It would still be a suboptimal shooting solution. Especially so if you plan to have it as your only piece of kit, and let's see why is that.
So, your argument is you only need a compact with a nice selection of lenses. Fine. This, in theory, makes you a serious enthusiast (it certainly does not make you a pro). The only difference between a serious enthusiast and a pro should be that the enthusiast does not make (enough) money out of photography so cost is a consideration, often a serious one. Implied here is that a serious enthusiast cares about image quality at least as much as a pro.
Does that describe you? Yes? Perfect. Then, go and buy a DSLR, too. A small sensor one is fine, too. Just don't tell me that you can get all the shots you want, exactly as you want them with just a compact - and do it all the time, to boot. Of course you can't. So, you need both (at least, a nice even smaller digital camera, like Sony DSC-RX100 may come handy, too). And if you have both, you will probably eventually find that faffing around with an unbalanced compact may not be much better than taking out your DSLR, unless you shoot wide or normal primes. In which case I point you back to my argument that an Fujifilm X100 may be a better choice anyway.
Did I persuade you? Probably not. Do I care too much? Probably not. I am actually hoping you'll go out and buy all the compact system camera and lenses you want. This will nicely fund the development of other (more) worthwhile photography kit - which can only be a good thing. Where I win (and you do, too, only I get to win for free).
Heck, if you do buy all that kit, and then find I was right after all, you may be inclined to sell it cheaply second hand. That's where I come again and buy it - to add to a DSLR, an X100, and an RX100. Win, win, win, win!