First: do you need to run Emacs on that system? Emacs can access remote files easily. Typically you just run it on your local machine. If you only ever access the system remotely, it probably doesn't need Emacs. ARM covers a very wide range of system sizes, from appliances with only a few megabytes of memory to equivalents of a 10-year old PC such as the latest Raspberry Pi model.
Second: do you need to compile Emacs yourself? There are plenty of Linux distributions that have ARM binaries and include Emacs. Distributions intended for smaller systems usually don't have it, but see “First” above. On “larger” distributions such as Debian or Arch, just install the package.
If you determine that you do need to compile Emacs, then you have a choice of cross-compiling or not. Emacs is written in C, and C compilers can work in relatively small amounts of memory. For example, you could compile on something like a Pi — but even a slightly aging PC will do it faster. If the device is powerful enough to cross-compile, then you'll probably want to run a distribution that has an Emacs package anyway.
If you do want to cross-compile, you'll need a cross-compiler. This can be a bit painful to set up sometimes. However, the distribution you're running on the device probably already has a cross-compiler setup. This is your most promising route: use whatever toolchain the distribution uses for its own packages.
Regarding Emacs itself, there are a few instructions in the
INSTALL file. You'll need to pass the
--host option to
./configure. The argument depends on the exact platform, depending on whether it has floating point registers and which standard library it runs. It might look something like
arm-linux-gnueabihf (with GNU libc and Hardware Floating point). Usually the argument is the same name that the toolchain is installed under; for example a compiler for the
arm-linux-gnueabihf target is typically called