Re: [arts-users] Fwd: Direct radiation is subtracted when it should not

2022-10-19 Thread Xia Mengqi
Hi Freddy,


Thanks for getting back to me. My question is about the result of using a 
specular reflective surface versus using a Lambertian surface. I was thinking 
the ratio should be pi (the specular one is larger) because when there is no 
scattering the only difference is the BRDF evaluation. However, I observed the 
Lambertian result is much smaller, more than 1e4 times smaller than the 
specular result. I tried "Test_iySurfaceLambertian.py" and it is also the case 
there when I change the Lambertian surface to specular.


Thanks,

Mengqi


From: Manfred Brath 
Sent: Wednesday, October 19, 2022 5:19:20 PM
To: Xia Mengqi; arts_users.mi@lists.uni-hamburg.de
Subject: Re: Fwd: [arts-users] Direct radiation is subtracted when it should not


Hi Mengqi,


I do not understand your problem from the information of your email, but maybe 
you can have a look in "Test_iySurfaceLambertian.py", which you can find in 
your arts-folder:


controlfiles-python/artscomponents/surface/


Another thing, how do you calculate your pi-ratio? When I use the above 
mentioned test script and set the sun at 0 lat and 0 lon and divide the top of 
the atmosphere (TOA) solar radiation F by pi I got the same value as from yCalc.


F=ws.stars.value[0].spectrum.value*scale_factor

scale_factor=sun_radius**2/(sun_radius**2+sun2TOA_distance**2)


The scale factor is needed, because the star irradiance spectrum in ARTS is 
defined at the surface of the sun and not at TOA.


Cheers,

Freddy


Am 18.10.22 um 12:56 schrieb Xia Mengqi:

Thank you Freddy! This makes sense.


I tried with flat reflective surface and lambertian surface with absorption 
only. I extended the 1D atmosphere to 3D so I can provide surface temperature 
(set it to a really small value) but in theory since there is no scattering the 
path that contributes is unchanged. I found that the flat reflective surface 
result is correct but the lambertian one is much smaller than the expected pi 
ratio. I tried to print ppath and I found los changing quite a lot. I am 
wondering if this is the expected behavior and maybe there is something about 
the 3D setup I did not understand properly. I copied the main part of the code 
below.


Thanks!

Mengqi


@arts_agenda
def propmat_clearsky_agenda(ws):
ws.Ignore(ws.rtp_mag)
ws.Ignore(ws.rtp_los)
ws.propmat_clearskyInit()
ws.propmat_clearskyAddConts()
ws.propmat_clearskyAddLines()

@arts_agenda
def gas_scattering_agenda(ws):
ws.Ignore(ws.rtp_vmr)
ws.gas_scattering_coefXsecConst(ConstXsec=4.65e-31)
ws.gas_scattering_matIsotropic()

# surface scattering agenda
# lambertian
@arts_agenda
def iy_surface_agenda(ws):
ws.iySurfaceInit()
ws.Ignore(ws.dsurface_rmatrix_dx)
ws.Ignore(ws.dsurface_emission_dx)

ws.iySurfaceLambertian()
ws.iySurfaceLambertianDirect()

# flat reflective surface
# @arts_agenda
# def iy_surface_agenda(ws):
# ws.iySurfaceInit()
# ws.iySurfaceFlatReflectivity()
# ws.iySurfaceFlatReflectivityDirect()

# generate atmosphere data
dataset_path = '/home/mandy/Github/arts/build_new/afgl_1986-us_standard.nc'
save_path = '/home/mandy/Github/arts/controlfiles/testdata/'
data = generate_atmos_arts(dataset_path, save_path)


# =
# open workspace
# =

ws = Workspace()
ws.verbositySetScreen(level=2)

# =
# generate atmosphere data
# =
dataset_path = '/home/mandy/Github/arts/build_new/afgl_1986-us_standard.nc'
save_path = '/home/mandy/Github/arts/controlfiles/testdata/'
data = generate_atmos_arts(dataset_path, save_path)

ws.ReadHITRAN(filename='/home/mandy/Github/MiAtmosphere/HITRAN/ALL.par', 
hitran_type="Online", abs_lines=ws.abs_lines)

# =
# select/define agendas
# =

ws.LegacyContinuaInit()
ws.PlanetSet(option="Earth")

# cosmic background radiation
ws.iy_space_agendaSet( option="CosmicBackground" )

# sensor-only path
ws.ppath_agendaSet( option="FollowSensorLosPath" )

# no refraction
ws.ppath_step_agendaSet( option="GeometricPath" )

# main agenda
ws.iy_main_agendaSet( option="Clearsky")

# water agenda
ws.water_p_eq_agendaSet()

# surface agenda
ws.iy_surface_agenda = iy_surface_agenda

ws.ArrayOfStringSet( ws.iy_aux_vars,
[ "Optical depth",
"Radiative background"
] )

ws.propmat_clearsky_agenda=propmat_clearsky_agenda

# gas scattering agenda
ws.gas_scattering_agenda = gas_scattering_agenda

ws.NumericSet( ws.ppath_lmax, 1e10)

# =
# basic conditions
# ===

Re: [arts-users] Fwd: Direct radiation is subtracted when it should not

2022-10-19 Thread Manfred Brath

Hi Mengqi,


I do not understand your problem from the information of your email, but 
maybe you can have a look in "Test_iySurfaceLambertian.py", which you 
can find in your arts-folder:



controlfiles-python/artscomponents/surface/


Another thing, how do you calculate your pi-ratio? When I use the above 
mentioned test script and set the sun at 0 lat and 0 lon and divide the 
top of the atmosphere (TOA) solar radiation F by pi I got the same value 
as from yCalc.


    F=ws.stars.value[0].spectrum.value*scale_factor

scale_factor=sun_radius**2/(sun_radius**2+sun2TOA_distance**2)


    The scale factor is needed, because the star irradiance spectrum in 
ARTS is defined at the surface of the sun and not at TOA.



Cheers,

Freddy


Am 18.10.22 um 12:56 schrieb Xia Mengqi:


Thank you Freddy! This makes sense.


I tried with flat reflective surface and lambertian surface with 
absorption only. I extended the 1D atmosphere to 3D so I can provide 
surface temperature (set it to a really small value) but in theory 
since there is no scattering the path that contributes is unchanged. I 
found that the flat reflective surface result is correct but the 
lambertian one is much smaller than the expected pi ratio. I tried to 
print ppath and I found los changing quite a lot. I am wondering if 
this is the expected behavior and maybe there is something about the 
3D setup I did not understand properly. I copied the main part of the 
code below.



Thanks!

Mengqi



@arts_agenda
def propmat_clearsky_agenda(ws):
    ws.Ignore(ws.rtp_mag)
    ws.Ignore(ws.rtp_los)
    ws.propmat_clearskyInit()
    ws.propmat_clearskyAddConts()
    ws.propmat_clearskyAddLines()

@arts_agenda
def gas_scattering_agenda(ws):
    ws.Ignore(ws.rtp_vmr)
    ws.gas_scattering_coefXsecConst(ConstXsec=4.65e-31)
    ws.gas_scattering_matIsotropic()

# surface scattering agenda
# lambertian
@arts_agenda
def iy_surface_agenda(ws):
    ws.iySurfaceInit()
    ws.Ignore(ws.dsurface_rmatrix_dx)
    ws.Ignore(ws.dsurface_emission_dx)

    ws.iySurfaceLambertian()
    ws.iySurfaceLambertianDirect()

# flat reflective surface
# @arts_agenda
# def iy_surface_agenda(ws):
#     ws.iySurfaceInit()
#     ws.iySurfaceFlatReflectivity()
#     ws.iySurfaceFlatReflectivityDirect()

# generate atmosphere data
dataset_path = 
'/home/mandy/Github/arts/build_new/afgl_1986-us_standard.nc'

save_path = '/home/mandy/Github/arts/controlfiles/testdata/'
data = generate_atmos_arts(dataset_path, save_path)


# 
=

# open workspace
# 
=


ws = Workspace()
ws.verbositySetScreen(level=2)

# 
=

# generate atmosphere data
# 
=
dataset_path = 
'/home/mandy/Github/arts/build_new/afgl_1986-us_standard.nc'

save_path = '/home/mandy/Github/arts/controlfiles/testdata/'
data = generate_atmos_arts(dataset_path, save_path)
ws.ReadHITRAN(filename='/home/mandy/Github/MiAtmosphere/HITRAN/ALL.par', 
hitran_type="Online", abs_lines=ws.abs_lines)


# 
=

# select/define agendas
# 
=


ws.LegacyContinuaInit()
ws.PlanetSet(option="Earth")

# cosmic background radiation
ws.iy_space_agendaSet( option="CosmicBackground" )

# sensor-only path
ws.ppath_agendaSet( option="FollowSensorLosPath" )

# no refraction
ws.ppath_step_agendaSet( option="GeometricPath" )

# main agenda
ws.iy_main_agendaSet( option="Clearsky")

# water agenda
ws.water_p_eq_agendaSet()

# surface agenda
ws.iy_surface_agenda = iy_surface_agenda

ws.ArrayOfStringSet( ws.iy_aux_vars,
[ "Optical depth",
"Radiative background"
] )

ws.propmat_clearsky_agenda=propmat_clearsky_agenda

# gas scattering agenda
ws.gas_scattering_agenda = gas_scattering_agenda

ws.NumericSet( ws.ppath_lmax, 1e10)

# 
=

# basic conditions
# 
=

# Postion and line-of-sight of sensor
sensor_pos = np.array([[600e+3, 0., 0.]])
# Sensor looking direction in zenith angle (0 = upwards, 180 = 
downward) and

# azimuth angle ( 0 = North, 90 = east)
sensor_los = np.array([[180, 0]])
ws.sensor_pos = sensor_pos
ws.sensor_los = sensor_los
ws.VectorSet(ws.rte_pos2, [])

# define environment
# 
=

# Number of Stokes components to be computed
ws.IndexSet(ws.stokes_dim, 1)

# Read the spectroscopic line data from the ARTS catalogue and
# create the workspace variable `lines'.
ws.ReadHITRAN(filename='/home/mandy/Github/MiAtmosphere/HITRAN/ALL.par', 
hitran_type="Online", abs_lines=ws.abs_lines)

ws.abs

[arts-users] Does the current retrieval pipeline work with star and scattering?

2022-10-19 Thread Xia Mengqi
Dear ARTS team,


I'm interested in retrieval problems and I'm wondering if it's possible to use 
the current retrieval pipeline with star and gas scattering together. It seems 
that in TestOEM.arts we only have gas absorption and the emission agenda.


Thanks,

Mengqi