laforge has submitted this change. ( https://gerrit.osmocom.org/c/pysim/+/36930?usp=email )
Change subject: CardKeyProvider: Implement support for column-based transport key encryption ...................................................................... CardKeyProvider: Implement support for column-based transport key encryption It's generally a bad idea to keep [card specific] key material lying around unencrypted in CSV files. The industry standard solution in the GSMA is a so-called "transport key", which encrypts the key material. Let's introduce support for this in the CardKeyProvider (and specifically, the CardKeyProviderCSV) and allow the user to specify transport key material as command line options to pySim-shell. Different transport keys can be used for different key materials, so allow specification of keys on a CSV-column base. The higher-level goal is to allow the CSV file not only to store the ADM keys (like now), but also global platform key material for establishing SCP towards various security domains in a given card. Change-Id: I13146a799448d03c681dc868aaa31eb78b7821ff --- A contrib/csv-encrypt-columns.py M docs/card-key-provider.rst M pySim-shell.py M pySim/card_key_provider.py 4 files changed, 192 insertions(+), 15 deletions(-) Approvals: fixeria: Looks good to me, but someone else must approve Jenkins Builder: Verified osmith: Looks good to me, approved diff --git a/contrib/csv-encrypt-columns.py b/contrib/csv-encrypt-columns.py new file mode 100755 index 0000000..2b2bbf3 --- /dev/null +++ b/contrib/csv-encrypt-columns.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 + +# Utility program to perform column-based encryption of a CSV file holding SIM/UICC +# related key materials. +# +# (C) 2024 by Harald Welte <lafo...@osmocom.org> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +import sys +import csv +import argparse +from Cryptodome.Cipher import AES + +from pySim.utils import h2b, b2h, Hexstr +from pySim.card_key_provider import CardKeyProviderCsv + +def dict_keys_to_upper(d: dict) -> dict: + return {k.upper():v for k,v in d.items()} + +class CsvColumnEncryptor: + def __init__(self, filename: str, transport_keys: dict): + self.filename = filename + self.transport_keys = dict_keys_to_upper(transport_keys) + + def encrypt_col(self, colname:str, value: str) -> Hexstr: + key = self.transport_keys[colname] + cipher = AES.new(h2b(key), AES.MODE_CBC, CardKeyProviderCsv.IV) + return b2h(cipher.encrypt(h2b(value))) + + def encrypt(self) -> None: + with open(self.filename, 'r') as infile: + cr = csv.DictReader(infile) + cr.fieldnames = [field.upper() for field in cr.fieldnames] + + with open(self.filename + '.encr', 'w') as outfile: + cw = csv.DictWriter(outfile, dialect=csv.unix_dialect, fieldnames=cr.fieldnames) + cw.writeheader() + + for row in cr: + for key_colname in self.transport_keys: + if key_colname in row: + row[key_colname] = self.encrypt_col(key_colname, row[key_colname]) + cw.writerow(row) + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument('CSVFILE', help="CSV file name") + parser.add_argument('--csv-column-key', action='append', required=True, + help='per-CSV-column AES transport key') + + opts = parser.parse_args() + + csv_column_keys = {} + for par in opts.csv_column_key: + name, key = par.split(':') + csv_column_keys[name] = key + + if len(csv_column_keys) == 0: + print("You must specify at least one key!") + sys.exit(1) + + csv_column_keys = CardKeyProviderCsv.process_transport_keys(csv_column_keys) + for name, key in csv_column_keys.items(): + print("Encrypting column %s using AES key %s" % (name, key)) + + cce = CsvColumnEncryptor(opts.CSVFILE, csv_column_keys) + cce.encrypt() diff --git a/docs/card-key-provider.rst b/docs/card-key-provider.rst index 82f2d65..7b4bf2c 100644 --- a/docs/card-key-provider.rst +++ b/docs/card-key-provider.rst @@ -41,6 +41,38 @@ open a CSV file from the default location at `~/.osmocom/pysim/card_data.csv`, and use that, if it exists. +Column-Level CSV encryption +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +pySim supports column-level CSV encryption. This feature will make sure +that your key material is not stored in plaintext in the CSV file. + +The encryption mechanism uses AES in CBC mode. You can use any key +length permitted by AES (128/192/256 bit). + +Following GSMA FS.28, the encryption works on column level. This means +different columns can be decrypted using different key material. This +means that leakage of a column encryption key for one column or set of +columns (like a specific security domain) does not compromise various +other keys that might be stored in other columns. + +You can specify column-level decryption keys using the +`--csv-column-key` command line argument. The syntax is +`FIELD:AES_KEY_HEX`, for example: + +`pySim-shell.py --csv-column-key SCP03_ENC_ISDR:000102030405060708090a0b0c0d0e0f` + +In order to avoid having to repeat the column key for each and every +column of a group of keys within a keyset, there are pre-defined column +group aliases, which will make sure that the specified key will be used +by all columns of the set: + +* `UICC_SCP02` is a group alias for `UICC_SCP02_KIC1`, `UICC_SCP02_KID1`, `UICC_SCP02_KIK1` +* `UICC_SCP03` is a group alias for `UICC_SCP03_KIC1`, `UICC_SCP03_KID1`, `UICC_SCP03_KIK1` +* `SCP03_ECASD` is a group alias for `SCP03_ENC_ECASD`, `SCP03_MAC_ECASD`, `SCP03_DEK_ECASD` +* `SCP03_ISDA` is a group alias for `SCP03_ENC_ISDA`, `SCP03_MAC_ISDA`, `SCP03_DEK_ISDA` +* `SCP03_ISDR` is a group alias for `SCP03_ENC_ISDR`, `SCP03_MAC_ISDR`, `SCP03_DEK_ISDR` + Field naming ------------ diff --git a/pySim-shell.py b/pySim-shell.py index 26f3d9b..7523ca2 100755 --- a/pySim-shell.py +++ b/pySim-shell.py @@ -967,6 +967,8 @@ help='script with pySim-shell commands to be executed automatically at start-up') global_group.add_argument('--csv', metavar='FILE', default=None, help='Read card data from CSV file') +global_group.add_argument('--csv-column-key', metavar='FIELD:AES_KEY_HEX', default=[], action='append', + help='per-CSV-column AES transport key') global_group.add_argument("--card_handler", dest="card_handler_config", metavar="FILE", help="Use automatic card handling machine") @@ -993,13 +995,18 @@ print("Invalid script file!") sys.exit(2) + csv_column_keys = {} + for par in opts.csv_column_key: + name, key = par.split(':') + csv_column_keys[name] = key + # Register csv-file as card data provider, either from specified CSV # or from CSV file in home directory csv_default = str(Path.home()) + "/.osmocom/pysim/card_data.csv" if opts.csv: - card_key_provider_register(CardKeyProviderCsv(opts.csv)) + card_key_provider_register(CardKeyProviderCsv(opts.csv, csv_column_keys)) if os.path.isfile(csv_default): - card_key_provider_register(CardKeyProviderCsv(csv_default)) + card_key_provider_register(CardKeyProviderCsv(csv_default, csv_column_keys)) # Init card reader driver sl = init_reader(opts, proactive_handler = Proact()) diff --git a/pySim/card_key_provider.py b/pySim/card_key_provider.py index 33a2a3d..6751b09 100644 --- a/pySim/card_key_provider.py +++ b/pySim/card_key_provider.py @@ -10,10 +10,10 @@ operation with pySim-shell. """ -# (C) 2021 by Sysmocom s.f.m.c. GmbH +# (C) 2021-2024 by Sysmocom s.f.m.c. GmbH # All Rights Reserved # -# Author: Philipp Maier +# Author: Philipp Maier, Harald Welte # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -29,18 +29,29 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. from typing import List, Dict, Optional +from Cryptodome.Cipher import AES +from pySim.utils import h2b, b2h import abc import csv card_key_providers = [] # type: List['CardKeyProvider'] +# well-known groups of columns relate to a given functionality. This avoids having +# to specify the same transport key N number of times, if the same key is used for multiple +# fields of one group, like KIC+KID+KID of one SD. +CRYPT_GROUPS = { + 'UICC_SCP02': ['UICC_SCP02_KIC1', 'UICC_SCP02_KID1', 'UICC_SCP02_KIK1'], + 'UICC_SCP03': ['UICC_SCP03_KIC1', 'UICC_SCP03_KID1', 'UICC_SCP03_KIK1'], + 'SCP03_ISDR': ['SCP03_ENC_ISDR', 'SCP03_MAC_ISDR', 'SCP03_DEK_ISDR'], + 'SCP03_ISDA': ['SCP03_ENC_ISDR', 'SCP03_MAC_ISDA', 'SCP03_DEK_ISDA'], + 'SCP03_ECASD': ['SCP03_ENC_ECASD', 'SCP03_MAC_ECASD', 'SCP03_DEK_ECASD'], + } class CardKeyProvider(abc.ABC): """Base class, not containing any concrete implementation.""" - VALID_FIELD_NAMES = ['ICCID', 'ADM1', - 'IMSI', 'PIN1', 'PIN2', 'PUK1', 'PUK2'] + VALID_KEY_FIELD_NAMES = ['ICCID', 'EID', 'IMSI' ] # check input parameters, but do nothing concrete yet def _verify_get_data(self, fields: List[str] = [], key: str = 'ICCID', value: str = "") -> Dict[str, str]: @@ -53,14 +64,10 @@ Returns: dictionary of {field, value} strings for each requested field from 'fields' """ - for f in fields: - if f not in self.VALID_FIELD_NAMES: - raise ValueError("Requested field name '%s' is not a valid field name, valid field names are: %s" % - (f, str(self.VALID_FIELD_NAMES))) - if key not in self.VALID_FIELD_NAMES: + if key not in self.VALID_KEY_FIELD_NAMES: raise ValueError("Key field name '%s' is not a valid field name, valid field names are: %s" % - (key, str(self.VALID_FIELD_NAMES))) + (key, str(self.VALID_KEY_FIELD_NAMES))) return {} @@ -84,19 +91,47 @@ class CardKeyProviderCsv(CardKeyProvider): - """Card key provider implementation that allows to query against a specified CSV file""" + """Card key provider implementation that allows to query against a specified CSV file. + Supports column-based encryption as it is generally a bad idea to store cryptographic key material in + plaintext. Instead, the key material should be encrypted by a "key-encryption key", occasionally also + known as "transport key" (see GSMA FS.28).""" + IV = b'\x23' * 16 csv_file = None filename = None - def __init__(self, filename: str): + def __init__(self, filename: str, transport_keys: dict): """ Args: filename : file name (path) of CSV file containing card-individual key/data + transport_keys : a dict indexed by field name, whose values are hex-encoded AES keys for the + respective field (column) of the CSV. This is done so that different fields + (columns) can use different transport keys, which is strongly recommended by + GSMA FS.28 """ self.csv_file = open(filename, 'r') if not self.csv_file: raise RuntimeError("Could not open CSV file '%s'" % filename) self.filename = filename + self.transport_keys = self.process_transport_keys(transport_keys) + + @staticmethod + def process_transport_keys(transport_keys: dict): + """Apply a single transport key to multiple fields/columns, if the name is a group.""" + new_dict = {} + for name, key in transport_keys.items(): + if name in CRYPT_GROUPS: + for field in CRYPT_GROUPS[name]: + new_dict[field] = key + else: + new_dict[name] = key + return new_dict + + def _decrypt_field(self, field_name: str, encrypted_val: str) -> str: + """decrypt a single field, if we have a transport key for the field of that name.""" + if not field_name in self.transport_keys: + return encrypted_val + cipher = AES.new(h2b(self.transport_keys[field_name]), AES.MODE_CBC, self.IV) + return b2h(cipher.decrypt(h2b(encrypted_val))) def get(self, fields: List[str], key: str, value: str) -> Dict[str, str]: super()._verify_get_data(fields, key, value) @@ -113,7 +148,7 @@ if row[key] == value: for f in fields: if f in row: - rc.update({f: row[f]}) + rc.update({f: self._decrypt_field(f, row[f])}) else: raise RuntimeError("CSV-File '%s' lacks column '%s'" % (self.filename, f)) -- To view, visit https://gerrit.osmocom.org/c/pysim/+/36930?usp=email To unsubscribe, or for help writing mail filters, visit https://gerrit.osmocom.org/settings Gerrit-Project: pysim Gerrit-Branch: master Gerrit-Change-Id: I13146a799448d03c681dc868aaa31eb78b7821ff Gerrit-Change-Number: 36930 Gerrit-PatchSet: 6 Gerrit-Owner: laforge <lafo...@osmocom.org> Gerrit-Reviewer: Jenkins Builder Gerrit-Reviewer: dexter <pma...@sysmocom.de> Gerrit-Reviewer: fixeria <vyanits...@sysmocom.de> Gerrit-Reviewer: laforge <lafo...@osmocom.org> Gerrit-Reviewer: osmith <osm...@sysmocom.de> Gerrit-MessageType: merged
