This Terms of Use explains the operation principle of the Content Lockers placed on this website.
On this website, you can encounter the Content Lockers which may ask
you to sign in, subscribe, enter your name or perform other actions to get access to the locked content.
When you enter your email or sign in through social networks, you agree to that your email address will be added to the subscription list for sending target news and special offers.
You can unsubscribe at any time by clicking on the link at the end of any of emails received from us.
When you sign in through social networks, the Content Locker may ask you to grant permissions to read or perform some social actions.
The Content Locker retrieves only the following information (according the Privacy Policy of this website):
Content Locker never collects other data and never publish anything in social networks from your behalf without your permissions.
After unlocking the content the Content Locker removes all the access tokens received from you and never uses them again.
If there are any questions regarding this Terms of Use you may contact us.


Usage: SIR-derived models
Here, we will create example datasets with simulated values of SIR-derived models. Then, we will perform scenario analysis with them.
Preparation
Prepare tha packages.
[1]:
# !pip install covsirphy --upgrade
from pprint import pprint
import covsirphy as cs
cs.__version__
[1]:
'2.22.0-beta'
Create example dataset with theoretical values
We will use ExampleData
class to perform simulation with preset initial values and parameters. \(\tau\) (coeficient for non-dimensionalization) will be set as \(1440\ \mathrm{[min]}\). The first date of records will be 01Jan2020 as an example.
[2]:
# Set tau value and start date of records
example_data = cs.ExampleData(tau=1440, start_date="01Jan2020")
No records were registered at this time.
[3]:
# Check records
example_data.cleaned()
[3]:
| Date | Country | Province | Confirmed | Infected | Fatal | Recovered |
|---|
ExampleData
class is a child class of JHUData
. i.e. We can use the example data in scenario analysis. Example codes will be shown in “Scenario analysis with theoretical data” subsection.
[4]:
issubclass(cs.ExampleData, cs.JHUData)
[4]:
True
[5]:
isinstance(example_data, cs.JHUData)
[5]:
True
SIR model
Let’s start with the simlest SIR model proposed by Kermack, W. O., & McKendrick, A. G. (1927). “Susceptible people” may meet “Infected” persons and may be confirmed as “Infected”. “Infected” patients will move to “Recovered” compertment later.
Variables:
\(\mathrm{S}\): Susceptible (= Population - Confirmed)
\(\mathrm{I}\): Infected (=Confirmed - Recovered - Fatal)
\(\mathrm{R}\): Recovered or Fatal (= Recovered + Fatal)
Parameters:
\(\beta\): Effective contact rate \(\mathrm{[1/min]}\)
\(\gamma\): Recovery (+ Mortality) rate \(\mathrm{[1/min]}\)
Non-dimensional SIR model
To simplify the model, we will remove the units of the variables from the ODE model.
Set \((S, I, R) = N \times (x, y, z)\) and \((T, \beta, \gamma) = (\tau t, \tau^{-1} \rho, \tau^{-1} \sigma)\).
This results in the ODE
Where \(N\) is the total population and \(\tau\) is a coefficient ([min], is an integer to simplify).
The model name and preset of parameter and initial values are registed as class variables of SIR
class.
[6]:
# Model name
print(cs.SIR.NAME)
# Example parameter values
pprint(cs.SIR.EXAMPLE, compact=True)
SIR
{'param_dict': {'rho': 0.2, 'sigma': 0.075},
'population': 1000000,
'step_n': 180,
'y0_dict': {'Fatal or Recovered': 0, 'Infected': 1000, 'Susceptible': 999000}}
With the preset values, ExampleData
instance will produce a example data.
[7]:
model = cs.SIR
area = {"country": "Full", "province": model.NAME}
# Add records with SIR model
example_data.add(model, **area)
We can get example records with ExampleData.specialized()
method.
[8]:
# Records with model variables
df = example_data.specialized(model, **area)
df.head()
[8]:
| Date | Susceptible | Infected | Fatal or Recovered | |
|---|---|---|---|---|
| 0 | 2020-01-02 | 998787 | 1133 | 80 |
| 1 | 2020-01-03 | 998547 | 1283 | 170 |
| 2 | 2020-01-04 | 998273 | 1454 | 273 |
| 3 | 2020-01-05 | 997964 | 1647 | 389 |
| 4 | 2020-01-06 | 997614 | 1865 | 521 |
With covsirphy.line_plot()
function, figures will be shown (or saved when filename
argument was applied).
[9]:
# Line plot with the example data
cs.line_plot(df.set_index("Date"), title=f"Example data of {model.NAME} model", y_integer=True)
Reproduction number
Reproduction number of SIR model is defined as follows.
\(R_0\) (“R naught”) means “the average number of secondary infections caused by an infected host” (external link: Infection Modeling — Part 1). When \(x=\frac{1}{R_0}\), \(\frac{\mathrm{d}y}{\mathrm{d}t}=0\) (the number of infected cases does not change).
We can calculate reproduction number using .calc_r0()
method.
[10]:
# Calculate reproduction number
# Note: population value will be applied, but not used in calculation
param_dict = cs.SIR.EXAMPLE["param_dict"].copy()
model_instance = cs.SIR(population=100000, **param_dict)
r0 = model_instance.calc_r0()
print(f"Reproduction number of {model_instance.NAME} model: {r0}")
Reproduction number of SIR model: 2.67
SIR-D model
Because we are measuring the number of fatal cases and recovered cases separately, we can use two variables (“Recovered” and “Deaths”) instead of “Recovered + Deaths” in the mathematical model. We call this model as SIR-D model.
Variables:
\(\mathrm{S}\): Susceptible (= Population - Confirmed)
\(\mathrm{I}\): Infected (=Confirmed - Recovered - Fatal)
\(\mathrm{R}\): Recovered
\(\mathrm{D}\): Fatal
Parameters:
\(\alpha\): Mortality rate \(\mathrm{[1/min]}\)
\(\beta\): Effective contact rate \(\mathrm{[1/min]}\)
\(\gamma\): Recovery rate \(\mathrm{[1/min]}\)
Non-dimensional SIR-D model
Set \((S, I, R, D) = N \times (x, y, z, w)\) and \((T, \alpha, \beta, \gamma) = (\tau t, \tau^{-1} \kappa, \tau^{-1} \rho, \tau^{-1} \sigma)\). This results in the ODE
The model name and preset values are registered in SIRD
class.
[11]:
# Model name
print(cs.SIRD.NAME)
# Example parameter values
pprint(cs.SIRD.EXAMPLE, compact=True)
SIR-D
{'param_dict': {'kappa': 0.005, 'rho': 0.2, 'sigma': 0.075},
'population': 1000000,
'step_n': 180,
'y0_dict': {'Fatal': 0,
'Infected': 1000,
'Recovered': 0,
'Susceptible': 999000}}
Example data is here.
[12]:
model = cs.SIRD
area = {"country": "Full", "province": model.NAME}
# Add records with SIR model
example_data.add(model, **area)
# Records with model variables
df = example_data.specialized(model, **area)
cs.line_plot(df.set_index("Date"), title=f"Example data of {model.NAME} model", y_integer=True)
Reproduction number
Reproduction number of SIR-D model is defined as follows.
We can calculate reproduction number using .calc_r0()
method.
[13]:
# Calculate reproduction number
# Note: population value will be applied, but not used in calculation
param_dict = cs.SIRD.EXAMPLE["param_dict"].copy()
model_instance = cs.SIRD(population=100000, **param_dict)
r0 = model_instance.calc_r0()
print(f"Reproduction number of {model_instance.NAME} model: {r0}")
Reproduction number of SIR-D model: 2.5
SIR-F model
In the initial phase of COVID-19 outbreak, many cases were confirmed after they died. To consider this issue, “S + I \(\to\) Fatal + I” should be added. We call the next model as SIR-F model. This is an original model of CovsirPhy. When \(\alpha_{1}=0\), no difference with the SIR-D model.
Variables:
\(\mathrm{S}\): Susceptible (= Population - Confirmed)
\(\mathrm{S}^\ast\): Confirmed and un-categorized
\(\mathrm{I}\): Confirmed and categorized as Infected
\(\mathrm{R}\): Confirmed and categorized as Recovered
\(\mathrm{F}\): Confirmed and categorzied as Fatal
Parameters:
\(\alpha_1\): Direct fatality probability of \(\mathrm{S}^\ast\) (non-dimensional)
\(\alpha_2\): Mortality rate of Infected cases \(\mathrm{[1/min]}\)
\(\beta\): Effective contact rate \(\mathrm{[1/min]}\)
\(\gamma\): Recovery rate \(\mathrm{[1/min]}\)
In JHU-style dataset, we know the number of cases who were confirmed with COVID-19, but we do not know the number of died cases who died without COVID-19. Essentially \(\mathrm{S}^\ast\) serves as an auxiliary compartment in SIR-F model to separate the two death situations and insert a probability factor of {\(\alpha_1\), \(1 - \alpha_1\)}.
SIR-F model was developed with Kaggle: COVID-19 data with SIR model.
Non-dimensional SIR-F model
Set \((S, I, R, F) = N \times (x, y, z, w)\) and \((T, \alpha_1, \alpha_2, \beta, \gamma) = (\tau t, \theta, \tau^{-1} \kappa, \tau^{-1} \rho, \tau^{-1} \sigma)\). This results in the ODE
We use SIRF
class.
[14]:
# Model name
print(cs.SIRF.NAME)
# Example parameter values
pprint(cs.SIRF.EXAMPLE, compact=True)
SIR-F
{'param_dict': {'kappa': 0.005, 'rho': 0.2, 'sigma': 0.075, 'theta': 0.002},
'population': 1000000,
'step_n': 180,
'y0_dict': {'Fatal': 0,
'Infected': 1000,
'Recovered': 0,
'Susceptible': 999000}}
Example data is here.
[15]:
model = cs.SIRF
area = {"country": "Full", "province": model.NAME}
# Add records with SIR model
example_data.add(model, **area)
# Records with model variables
df = example_data.specialized(model, **area)
cs.line_plot(df.set_index("Date"), title=f"Example data of {model.NAME} model", y_integer=True)
Reproduction number
Reproduction number of SIR-F model is defined as follows.
We can calculate reproduction number using .calc_r0()
method.
[16]:
# Calculate reproduction number
# Note: population value will be applied, but not used in calculation
param_dict = cs.SIRF.EXAMPLE["param_dict"].copy()
model_instance = cs.SIRF(population=100000, **param_dict)
r0 = model_instance.calc_r0()
print(f"Reproduction number of {model_instance.NAME} model: {r0}")
Reproduction number of SIR-F model: 2.5
SIR-F with exposed/waiting cases
The next model is SEWIR-F model. The number of exposed cases in latent period (E) and wating cases for confirmation (W) are un-measurable variables, but key variables as well as S, I, R, F. If E and W are large, outbreak will occur in the near future. Let’s replace S\(\overset{\beta I}{\longrightarrow}\)S\(^\ast\) as follows because W also has infectivity.
Variables:
\(\mathrm{S}\): Susceptible
\(\mathrm{E}\): Exposed and in latent period (without infectivity)
\(\mathrm{W}\): Waiting for confirmaion fiagnosis (with infectivity)
\(\mathrm{S}^\ast\): Confirmed and un-categorized
\(\mathrm{I}\): Confirmed and categorized as Infected
\(\mathrm{R}\): Confirmed and categorized as Recovered
\(\mathrm{F}\): Confirmed and categorzied as Fatal
Parameters:
\(\alpha_1\): Direct fatality probability of \(\mathrm{S}^\ast\) (non-dimensional)
\(\alpha_2\): Mortality rate of Infected cases \(\mathrm{[1/min]}\)
\(\beta_1\): Exposure rate (the nymber of encounter with the virus in a minute) \(\mathrm{[1/min]}\)
\(\beta_2\): Inverse of latent period \(\mathrm{[1/min]}\)
\(\beta_3\): Inverse of waiting time for confirmation \(\mathrm{[1/min]}\)
\(\gamma\): Recovery rate \(\mathrm{[1/min]}\)
Non-dimensional SEWIR-F model
Set \((S, E, W, I, R, F) = N \times (x_1, x_2, x_3, y, z, w)\), \((T, \alpha_1) = (\tau t, \theta)\) and \((\alpha_2, \beta_i, \gamma) = \tau^{-1} \times (\kappa, \rho_i, \sigma)\). This results in the ODE
SEWIRF
class is for the SEWIR-F model.
[17]:
# Model name
print(cs.SEWIRF.NAME)
# Example parameter values
pprint(cs.SEWIRF.EXAMPLE, compact=True)
SEWIR-F
{'param_dict': {'kappa': 0.005,
'rho1': 0.2,
'rho2': 0.167,
'rho3': 0.167,
'sigma': 0.075,
'theta': 0.002},
'population': 1000000,
'step_n': 180,
'y0_dict': {'Exposed': 3000,
'Fatal': 0,
'Infected': 1000,
'Recovered': 0,
'Susceptible': 994000,
'Waiting': 2000}}
Example records are here.
[18]:
model = cs.SEWIRF
area = {"country": "Full", "province": model.NAME}
# Add records with SIR model
example_data.add(model, **area)
# Records with model variables
df = example_data.specialized(model, **area)
cs.line_plot(df.set_index("Date"), title=f"Example data of {model.NAME} model", y_integer=True)
Reproduction number
Reproduction number of SEWIR-F model is defined as follows.
We can calculate reproduction number using .calc_r0()
method.
[19]:
# Calculate reproduction number
# Note: population value will be applied, but not used in calculation
param_dict = cs.SEWIRF.EXAMPLE["param_dict"].copy()
model_instance = cs.SEWIRF(population=100000, **param_dict)
r0 = model_instance.calc_r0()
print(f"Reproduction number of {model_instance.NAME} model: {r0}")
Reproduction number of SEWIR-F model: 2.49
SIR-F with vaccination
Vaccination is a key factor to prevent outbreak as you know.
In the previous version, we defined SIR-FV model with \(\omega\) (vaccination rate) and
However, SIR-FV model was removed because vaccinated persons may move to the other compartments, including “Susceprtible”. Please use SIR-F model for simulation and parameter estimation with adjusted parameter values, considering the impact of vaccinations on infectivity, its effectivity and safety.
SIR-F with re-infection
Re-infection (Recovered -> Susceptble) is sometimes reported and we can consider SIR-S (SIR-FS) model. However, this is not impremented at this time because we do not have data regarding re-infection. SIR-F model could be the final model in our data-driven approach at this time.
Re-infection changes the parameter values of SIR-F model. There are two patterns.
If re-infected case are counted as new confirmed cases and removed from “Recovered” compartment, \(\sigma\) will be decreased.
If re-infected cases are counted as new confirmed cases and NOT removed from “Recovered” compartment, \(\rho\) will be increased because “Susceptible” will be decreased.
Impact of parameter change
Because ExampleData
class is a subclass of JHUData
, we can perform scenario analysis with example datasets easily. We evaluate the impact of parameter changes.
Here, we will use the following scenarios. For explanation, \(\tau=1440\), the start date is 01Jan2020, population is 1,000,000 and country name is “Theoretical”. Their scenarios are not based on actual data.
name |
01Jan2020 - 31Jan2020 |
01Feb2020 - 31Dec2020 |
|---|---|---|
Main |
SIR-F |
SIR-F |
Lockdown |
SIR-F |
SIR-F with 50% of \(\rho\) |
Medicine |
SIR-F |
SIR-F with 50% of \(\kappa\) and 200% of \(\sigma\) |
Vaccine |
SIR-F |
SIR-F with 80% of \(\rho\), 60% of \(\kappa\) and 120% of \(\sigma\) |
As baseline (main scenario), we use preset values of the SIR-F model.
[20]:
# Preset of SIR-F parameters and initial values
preset_dict = cs.SIRF.EXAMPLE["param_dict"]
preset_dict
[20]:
{'theta': 0.002, 'kappa': 0.005, 'rho': 0.2, 'sigma': 0.075}
Create records from 01Jan2020 to 31Jan2020. These records will be used commonly in the scenarios.
[21]:
area = {"country": "Theoretical"}
# Create dataset from 01Jan2020 to 31Jan2020
example_data.add(cs.SIRF, step_n=30, **area)
Create Scenario
instance for scenario analysis.
[22]:
# Create Scenario instance
snl = cs.Scenario(tau=1440, **area)
snl.register(example_data)
Then, confirm the records with Scenario.records()
instance.
[23]:
# Show records with Scenario instance
record_df = snl.records()
display(record_df.head())
display(record_df.tail())
| Date | Infected | Fatal | Recovered | |
|---|---|---|---|---|
| 0 | 2020-01-02 | 1127 | 6 | 80 |
| 1 | 2020-01-03 | 1270 | 12 | 169 |
| 2 | 2020-01-04 | 1430 | 19 | 271 |
| 3 | 2020-01-05 | 1612 | 28 | 385 |
| 4 | 2020-01-06 | 1816 | 37 | 513 |
| Date | Infected | Fatal | Recovered | |
|---|---|---|---|---|
| 25 | 2020-01-27 | 21210 | 940 | 13068 |
| 26 | 2020-01-28 | 23730 | 1061 | 14753 |
| 27 | 2020-01-29 | 26524 | 1196 | 16637 |
| 28 | 2020-01-30 | 29616 | 1347 | 18741 |
| 29 | 2020-01-31 | 33030 | 1515 | 21088 |
Then, set the records from 02Jan2020 to 31Jan2020 as the 0th phase. The 0th phase is commonly used in the all scenarios.
[24]:
# Set 0th phase from 02Jan2020 to 31Jan2020 with preset parameter values
snl.clear(include_past=True)
snl.add(end_date="31Jan2020", model=cs.SIRF, **preset_dict)
# Show summary
snl.summary()
[24]:
| Type | Start | End | Population | ODE | Rt | theta | kappa | rho | sigma | tau | alpha1 [-] | 1/alpha2 [day] | 1/beta [day] | 1/gamma [day] | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0th | Past | 02Jan2020 | 31Jan2020 | 1000000 | SIR-F | 2.5 | 0.002 | 0.005 | 0.2 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 |
Set the 1st phase with the same parameter values.
[25]:
# Add main scenario
snl.add(end_date="31Dec2020", name="Main")
snl.summary()
[25]:
| Type | Start | End | Population | ODE | Rt | theta | kappa | rho | sigma | tau | alpha1 [-] | 1/alpha2 [day] | 1/beta [day] | 1/gamma [day] | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0th | Past | 02Jan2020 | 31Jan2020 | 1000000 | SIR-F | 2.5 | 0.002 | 0.005 | 0.2 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 |
| 1st | Future | 01Feb2020 | 31Dec2020 | 1000000 | SIR-F | 2.5 | 0.002 | 0.005 | 0.2 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 |
Copy the main scenario and name it as Lockdown scenario. Scenario.clear()
removes the future phase (th 1st phase here) and we will register th 1st phase with halved \(\rho\) value. Lockdown is supposed to reduce effective contact rate.
[26]:
# Add lockdown scenario
snl.clear(name="Lockdown")
# Get rho value of the 0th phase and halve it
rho_lock = snl.get("rho", phase="0th") * 0.5
# Add th 1st phase with the calculated rho value
snl.add(end_date="31Dec2020", name="Lockdown", rho=rho_lock)
[26]:
<covsirphy.analysis.scenario.Scenario at 0x7fce83cd5220>
Next, we define medicine scenario. \(\kappa\) will be halved and \(\sigma\) will be doubled. New medicines may reduce the severity rate and enhance recovery.
[27]:
# Add medicine scenario
snl.clear(name="Medicine")
kappa_med = snl.get("kappa", phase="0th") * 0.5
sigma_med = snl.get("sigma", phase="0th") * 2
snl.add(end_date="31Dec2020", name="Medicine", kappa=kappa_med, sigma=sigma_med)
[27]:
<covsirphy.analysis.scenario.Scenario at 0x7fce83cd5220>
We define vaccine scenario. As noted in “SIR-F model with vaccination” section, vaccination impacts on \(\sigma\) and \(\kappa\) with depending on its effectivity and safety. If vaccinations impact on infectivity, \(\rho\) value will be also changed.
[28]:
# Add vaccine scenario
snl.clear(name="Vaccine")
rho_vac = snl.get("rho", phase="0th") * 0.8
kappa_vac = snl.get("kappa", phase="0th") * 0.6
sigma_vac = snl.get("sigma", phase="0th") * 1.2
snl.add(end_date="31Dec2020", name="Vaccine", rho=rho_vac, kappa=kappa_vac, sigma=sigma_vac)
[28]:
<covsirphy.analysis.scenario.Scenario at 0x7fce83cd5220>
See the phase settings with Scenario.summary()
as always.
[29]:
# Show summary
snl.summary()
[29]:
| Type | Start | End | Population | ODE | Rt | theta | kappa | rho | sigma | tau | alpha1 [-] | 1/alpha2 [day] | 1/beta [day] | 1/gamma [day] | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Scenario | Phase | |||||||||||||||
| Main | 0th | Past | 02Jan2020 | 31Jan2020 | 1000000 | SIR-F | 2.50 | 0.002 | 0.0050 | 0.20 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 |
| 1st | Future | 01Feb2020 | 31Dec2020 | 1000000 | SIR-F | 2.50 | 0.002 | 0.0050 | 0.20 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 | |
| Lockdown | 0th | Past | 02Jan2020 | 31Jan2020 | 1000000 | SIR-F | 2.50 | 0.002 | 0.0050 | 0.20 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 |
| 1st | Future | 01Feb2020 | 31Dec2020 | 1000000 | SIR-F | 1.25 | 0.002 | 0.0050 | 0.10 | 0.075 | 1440 | 0.002 | 200.0 | 10.0 | 13.0 | |
| Medicine | 0th | Past | 02Jan2020 | 31Jan2020 | 1000000 | SIR-F | 2.50 | 0.002 | 0.0050 | 0.20 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 |
| 1st | Future | 01Feb2020 | 31Dec2020 | 1000000 | SIR-F | 1.31 | 0.002 | 0.0025 | 0.20 | 0.150 | 1440 | 0.002 | 400.0 | 5.0 | 6.0 | |
| Vaccine | 0th | Past | 02Jan2020 | 31Jan2020 | 1000000 | SIR-F | 2.50 | 0.002 | 0.0050 | 0.20 | 0.075 | 1440 | 0.002 | 200.0 | 5.0 | 13.0 |
| 1st | Future | 01Feb2020 | 31Dec2020 | 1000000 | SIR-F | 1.72 | 0.002 | 0.0030 | 0.16 | 0.090 | 1440 | 0.002 | 333.0 | 6.0 | 11.0 |
Show the parameter setting with Scenario.history()
.
[30]:
# Show the history of rho as a dataframe and a figure
# we can set theta/kappa/rho/sigma for SIR-F model
snl.history(target="rho").head()
[30]:
| Scenario | Lockdown | Main | Medicine | Vaccine |
|---|---|---|---|---|
| Date | ||||
| 2020-01-02 | 0.2 | 0.2 | 0.2 | 0.2 |
| 2020-01-03 | 0.2 | 0.2 | 0.2 | 0.2 |
| 2020-01-04 | 0.2 | 0.2 | 0.2 | 0.2 |
| 2020-01-05 | 0.2 | 0.2 | 0.2 | 0.2 |
| 2020-01-06 | 0.2 | 0.2 | 0.2 | 0.2 |
Compare the scenarios
We will compare the scenarios to discuss the impact of changing parameters. We have some methods for that.
Scenario.describe()
shows representative values as a dataframe.Scenario.history(target="Rt")
shows the history of automatically calculated reproduction number.Scenario.history(target="variable name")
shows simulated number of cases for the specified variable.If you have any ideas, please create issues! :)
1. Show representative values
Scenario.describe()
compares
max number of infected cases and the date with the max value,
the number of confirmed/infected/fatal cases on the next date of the last phase, and
reproduction numbers for the phases with different numbers.
[31]:
# Describe the scenarios
snl.describe()
[31]:
| max(Infected) | argmax(Infected) | Confirmed on 31Dec2020 | Infected on 31Dec2020 | Fatal on 31Dec2020 | 1st_Rt | |
|---|---|---|---|---|---|---|
| Main | 232800 | 02Mar2020 | 890043 | 0 | 57295 | 2.50 |
| Lockdown | 44360 | 12Mar2020 | 436501 | 137 | 28090 | 1.25 |
| Medicine | 51404 | 24Feb2020 | 483356 | 0 | 9911 | 1.31 |
| Vaccine | 113534 | 09Mar2020 | 708848 | 0 | 24917 | 1.72 |
2. History of reproduction number
Scenario.history(target="Rt")
shows the history of reproduction number.
[32]:
# Show the history of reproduction number
_ = snl.history(target="Rt")
3. Simulated number of cases of the specified variable
We can also set Confired/Infected/Fatal/Recovered as the target of Scenario.history()
.
[33]:
# The number of infected cases
_ = snl.history(target="Infected")
[34]:
# The number of fatal cases
_ = snl.history(target="Fatal")
Simulation of each scenario
We can simulate the all kind of the number of cases for a specific scenario with Scenario.simulate()
.
[35]:
# Main scenario
_ = snl.simulate(name="Main")
[36]:
# Lockdown scenario
_ = snl.simulate(name="Lockdown")
[37]:
# Medicine scenario
_ = snl.simulate(name="Medicine")
[38]:
# Vaccine scenario
_ = snl.simulate(name="Vaccine")
For futher analysis, let’s change the parameter settings and add new scenarios!


Click the button to set the href attribute with a value of "www.w3schools.com" of the a element above.
Note: Internet Explorer 8 and earlier does not support the setAttribute method.


@article{Agrawal95, author = {R. Agrawal}, journal = {{Advances in Applied Probability}}, number = {4}, pages = {1054--1078}, publisher = {{Cambridge University Press}}, title = {{Sample mean based index policies by $\mathcal{O}(\log n)$ regret for the Multi-Armed Bandit problem}}, volume = {27}, year = {1995} }
@article{AlaturLevyKrause19, author = {P. Alatur and K. Y. Levy and A. Krause}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1902.08036}{arXiv:1902.08036}}}, title = {{Multi-Player Bandits: The Adversarial Case}}, url = {https://arxiv.org/abs/1902.08036}, year = {2019} }
@article{Allesiardo17, author = {R. Allesiardo and R. F{\'e}raud and O.-A. Maillard}, journal = {{International Journal of Data Science and Analytics}}, number = {4}, pages = {267--283}, publisher = {Springer}, title = {{The Non-Stationary Stochastic Multi-Armed Bandit Problem}}, volume = {3}, year = {2017} }
@article{Anandkumar11, author = {A. Anandkumar and N. Michael and A. K. Tang and S. Agrawal}, journal = {Journal on Selected Areas in Communications}, number = {4}, pages = {731--745}, publisher = {IEEE}, title = {{Distributed Algorithms for Learning and Cognitive Medium Access with Logarithmic Regret}}, volume = {29}, year = {2011} }
@article{Anantharam87a, author = {V. Anantharam and P. Varaiya and J. Walrand}, journal = {{Transactions on Automatic Control}}, number = {11}, pages = {968--976}, publisher = {IEEE}, title = {{Asymptotically efficient allocation rules for the Multi-Armed Bandit problem with multiple plays - Part I: IID rewards}}, volume = {32}, year = {1987} }
@article{Anantharam87b, author = {V. Anantharam and P. Varaiya and J. Walrand}, journal = {{Transactions on Automatic Control}}, number = {11}, pages = {977--982}, publisher = {IEEE}, title = {{Asymptotically efficient allocation rules for the Multi-Armed Bandit problem with multiple plays - Part II: Markovian rewards}}, volume = {32}, year = {1987} }
@article{Arora2012multiplicative, author = {S. Arora and E. Hazan and S. Kale}, journal = {Theory of Computing}, number = {1}, pages = {121--164}, publisher = {Theory of Computing Exchange}, title = {The Multiplicative Weights Update Method: a Meta-Algorithm and Applications}, volume = {8}, year = {2012} }
@article{ArrowEnthoven61, author = {K. J. Arrow and A. C. Enthoven}, journal = {Econometrica}, number = {4}, pages = {779--800}, publisher = {Wiley, Econometric Society}, title = {{Quasi-Concave Programming}}, volume = {29}, year = {1961} }
@article{Auer02, author = {P. Auer and N. Cesa-Bianchi and P. Fischer}, doi = {10.1023/A:1013689704352}, journal = {Machine Learning}, number = {2}, pages = {235--256}, title = {{Finite-time Analysis of the Multi-armed Bandit Problem}}, volume = {47}, year = {2002} }
@article{Auer02NonStochastic, author = {P. Auer and N. Cesa-Bianchi and Y. Freund and R. Schapire}, journal = {SIAM journal on computing}, number = {1}, pages = {48--77}, publisher = {SIAM}, title = {{The Non-Stochastic Multi-Armed Bandit Problem}}, volume = {32}, year = {2002} }
@article{Auer10, author = {P. Auer and R. Ortner}, journal = {Periodica Mathematica Hungarica}, number = {1-2}, pages = {55--65}, publisher = {Akad{\'e}miai Kiad{\'o}}, title = {{UCB Revisited: Improved Regret Bounds For The Stochastic Multi-Armed Bandit Problem}}, volume = {61}, year = {2010} }
@article{AuerGajaneOrtner18, author = {P. Auer and P. Gajane and R. Ortner}, journal = {European Workshop on Reinforcement Learning}, title = {{Adaptively Tracking the Best Arm with an Unknown Number of Distribution Changes}}, url = {https://ewrl.files.wordpress.com/2018/09/ewrl_14_2018_paper_28.pdf}, year = {2018} }
@article{Avner15, author = {O. Avner and S. Mannor}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1504.08167}{arXiv:1504.08167}}}, title = {{Learning to Coordinate Without Communication in Multi-User Multi-Armed Bandit Problems}}, url = {https://arxiv.org/abs/1504.08167}, year = {2015} }
@article{AvnerMannor18, author = {O. Avner and S. Mannor}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1808.04875}{arXiv:1808.04875}}}, title = {{Multi-User Communication Networks: A Coordinated Multi-Armed Bandit Approach}}, url = {https://arxiv.org/abs/1808.04875}, year = {2018} }
@article{Baransi2014, author = {A. Baransi and O.-A. Maillard and S. Mannor}, journal = {{Proceedings of the European Conference on Machine Learning}}, title = {{Sub-sampling for Multi-armed Bandits}}, url = {https://hal.archives-ouvertes.fr/hal-01025651}, year = {2014} }
@article{Besson2019Gretsi, author = {L. Besson and E. Kaufmann}, howpublished = {Online at: \texttt{\url{https://hal.archives-ouvertes.fr/hal-02152243}}. Code at: \texttt{\url{https://SMPyBandits.GitHub.io/NonStationary.html}}}, journal = {GRETSI}, month = {August}, note = {\texttt{\url{https://hal.archives-ouvertes.fr/hal-02152243}}}, pdf = {https://perso.crans.org/besson/articles/BK__GRETSI_2019.pdf}, title = {{Analyse non asymptotique d'un test séquentiel de détection de ruptures et application aux bandits non stationnaires}}, url = {https://hal.archives-ouvertes.fr/hal-02152243}, year = {2019} }
@article{Bonnefoi18, author = {R. Bonnefoi and C. Moy and J. Palicot}, doi = {10.1186/s13638-018-1044-2}, journal = {EURASIP Journal on Wireless Communications and Networking}, number = {1}, pages = {34}, title = {{Improvement of the LPWAN AMI backhaul's latency thanks to reinforcement learning algorithms}}, volume = {2018}, year = {2018} }
@article{BoursierPerchet18, author = {E. Boursier and V. Perchet}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1809.08151}{arXiv:1809.08151}}}, title = {{SIC-MMAB: Synchronisation Involves Communication in Multiplayer Multi-Armed Bandits}}, url = {https://arxiv.org/abs/1809.08151}, year = {2018} }
@article{Bubeck12, author = {S. Bubeck and N. Cesa-Bianchi}, journal = {{Foundations and Trends{\textregistered} in Machine Learning}}, number = {1}, pages = {1--122}, publisher = {Now Publishers, Inc.}, title = {{Regret Analysis of Stochastic and Non-Stochastic Multi-Armed Bandit Problems}}, volume = {5}, year = {2012} }
@article{Burnetas96, author = {A. N. Burnetas and M. N. Katehakis}, journal = {Advances in Applied Mathematics}, number = {2}, pages = {122--142}, publisher = {Elsevier}, title = {{Optimal Adaptive Policies for Sequential Allocation Problems}}, volume = {17}, year = {1996} }
@article{Centenaro16, author = {M. Centenaro and L. Vangelista and A. Zanella and M. Zorzi}, doi = {10.1109/MWC.2016.7721743}, journal = {Wireless Communications}, number = {5}, pages = {60--67}, publisher = {IEEE}, title = {Long-range communications in unlicensed bands: the rising stars in the {IoT} and smart city scenarios}, volume = {23}, year = {2016} }
@article{Chapelleetal14Ad, author = {O. Chapelle and E. Manavoglu and R. Rosales}, journal = {Transactions on Intelligent Systems and Technology}, title = {{Simple and Scalable Response Prediction For Display Advertising}}, year = {2014} }
@article{Darak16, author = {S. J. Darak and C. Moy and J. Palicot}, journal = {{EAI Endorsed Transactions on Cognitive Communications}}, pages = {1--10}, title = {{Proof-of-Concept System for Opportunistic Spectrum Access in Multi-user Decentralized Networks}}, volume = {2}, year = {2016} }
@article{DarakHanawal18, author = {S. J. Darak and M. K. Hanawal}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1812.11651}{arXiv:1812.11651}}}, title = {{Distributed Learning and Stable Orthogonalization in Ad-Hoc Networks with Heterogeneous Channels}}, url = {https://arxiv.org/abs/1812.11651}, year = {2018} }
@article{GaBaVa18, author = {G. Gautier and R. Bardenet and M. Valko}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1809.07258}{arXiv:1809.07258}}}, keywords = {Computer Science - Machine Learning, Computer Science - Mathematical Software, Statistics - Machine Learning}, note = {Code at \texttt{\url{https://github.com/guilgautier/DPPy}}. Documentation at \texttt{\url{https://dppy.readthedocs.io}}}, title = {{DPPy: Sampling Determinantal Point Processes with Python}}, url = {https://arxiv.org/abs/1809.07258}, year = {2018} }
@article{Garivier16TrueShape, author = {A. Garivier and P. M{\'e}nard and G. Stoltz}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1602.07182}{arXiv:1602.07182}}}, title = {{Explore First, Exploit Next: The True Shape of Regret in Bandit Problems}}, url = {https://arxiv.org/abs/1602.07182}, year = {2016} }
@article{GarivierHadiji2018, author = {A. Garivier and H. Hadiji and P. Menard and G. Stoltz}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1805.05071}{arXiv:1805.05071}}}, title = {{KL-UCB-switch: optimal regret bounds for stochastic bandits from both a distribution-dependent and a distribution-free viewpoints}}, url = {https://arxiv.org/abs/1805.05071}, year = {2018} }
@article{Haykin05, author = {S. Haykin}, journal = {Journal on Selected Areas in Communications}, number = {2}, pages = {201--220}, publisher = {IEEE}, title = {{Cognitive Radio: Brain-Empowered Wireless Communications}}, volume = {23}, year = {2005} }
@article{Hazan2016introduction, author = {E. Hazan and others}, journal = {Foundations and Trends{\textregistered} in Optimization}, number = {3-4}, pages = {157--325}, publisher = {Now Publishers, Inc.}, title = {{Introduction to online convex optimization}}, volume = {2}, year = {2016} }
@article{Honda2019, author = {J. Honda}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1903.07839}{arXiv:1903.07839}}}, title = {{A Note on KL-UCB+ Policy for the Stochastic Bandit}}, url = {https://arxiv.org/abs/1903.07839}, year = {2019} }
@article{Jouini12, author = {W. Jouini and C. Moy and J. Palicot}, journal = {{EURASIP Journal on Wireless Communications and Networking}}, number = {1}, publisher = {Springer}, title = {{Decision Making for Cognitive Radio Equipment: Analysis of the First 10 Years of Exploration}}, volume = {2012}, year = {2012} }
@article{KK18Martingales, author = {E. Kaufmann and W. M. Koolen}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1811.11419}{arXiv:1811.11419}}}, title = {{Mixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervals}}, url = {https://arXiv.org/abs/1811.11419}, year = {2018} }
@article{KLUCBJournal, author = {O. Capp{\'e} and A. Garivier and O.-A. Maillard and R. Munos and G. Stoltz}, journal = {Annals of Statistics}, pages = {1516--1541}, title = {{Kullback-Leibler Upper Confidence Bounds For Optimal Sequential Allocation}}, volume = {41(3)}, year = {2013} }
@article{Kaufmann17, author = {E. Kaufmann and A. Garivier}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1702.00001}{arXiv:1702.00001}}}, title = {{Learning The Distribution With Largest Mean: Two Bandit Frameworks}}, url = {https://arxiv.org/abs/1702.00001}, year = {2017} }
@article{KaufmannAbbas19, author = {E. Kaufmann and A. Mehrabian}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1902.01239}{arXiv:1902.01239}}}, title = {{New Algorithms for Multiplayer Bandits when Arm Means Vary Among Players}}, url = {https://arxiv.org/abs/1902.01239}, year = {2019} }
@article{Keriven2018, author = {N. Keriven and G. Damien and P. Iacopo}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1805.08061}{arXiv:1805.08061}}}, note = {Code at \texttt{\url{https://github.com/lightonai/newma}}}, title = {{NEWMA: a new method for scalable model-free online change-point detection}}, url = {https://arxiv.org/abs/1805.08061}, year = {2018} }
@article{KimTewari2019, author = {B. Kim and A. Tewari}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1902.00610}{arXiv:1902.00610}}}, title = {{On the Optimality of Perturbations in Stochastic and Adversarial Multi-Armed Bandit Problems}}, url = {https://arxiv.org/abs/1902.00610}, year = {2019} }
@article{KullbackLeibler51, author = {S. Kullback and R.A. Leibler}, journal = {The Annals of Mathematical Statistics}, number = {1}, pages = {79--86}, publisher = {JSTOR}, title = {{On Information and Sufficiency}}, volume = {22}, year = {1951} }
@article{KumarDarak2019, author = {R. Kumar and S. J. Darak and M. K. Hanawal and A. K. Sharma and R. K. Tripathi}, doi = {10.1109/LCOMM.2018.2890226}, issn = {1089-7798}, journal = {IEEE Communications Letters}, number = {2}, pages = {362--365}, title = {{Distributed Algorithm for Learning to Coordinate in Infrastructure-Less Network}}, volume = {23}, year = {2019} }
@article{LaiRobbins85, author = {T. L. Lai and H. Robbins}, journal = {{Advances in Applied Mathematics}}, number = {1}, pages = {4--22}, title = {{Asymptotically Efficient Adaptive Allocation Rules}}, volume = {6}, year = {1985} }
@article{LaiXing10, author = {T. L. Lai and H. Xing}, journal = {Sequential Analysis}, number = {2}, pages = {162--175}, title = {{Sequential change-point detection when the pre-and post-change parameters are unknown}}, volume = {29}, year = {2010} }
@article{Lattimore16b, author = {T. Lattimore}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1603.08661}{arXiv:1603.08661}}}, title = {{Regret Analysis of the Anytime Optimally Confident UCB Algorithm}}, url = {https://arxiv.org/abs/1603.08661}, year = {2016} }
@article{Lattimore2018refining, author = {T. Lattimore}, journal = {The Journal of Machine Learning Research}, number = {1}, pages = {765--796}, publisher = {JMLR. org}, title = {{Refining the confidence level for optimistic bandit strategies}}, volume = {19}, year = {2018} }
@article{LiJamieson2018, author = {L. Li and K. Jamieson}, journal = {Journal of Machine Learning Research}, pages = {1--52}, title = {{Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization}}, url = {https://arxiv.org/abs/1603.06560}, volume = {18}, year = {2018} }
@article{Luedtke16, arxiv = {{arXiv preprint \href{https://arXiv.org/abs/1606.09388}{arXiv:1606.09388}}}, author = {A. Luedtke and E. Kaufmann and A. Chambaz}, journal = {Machine Learning}, oldtitle = {{Asymptotically Optimal Algorithms for Multiple Play Bandits with Partial Feedback}}, pages = {1--31}, publisher = {Springer}, title = {{Asymptotically Optimal Algorithms for Budgeted Multiple Play Bandits}}, url = {https://arxiv.org/abs/1606.09388}, year = {2017} }
@article{Luenberger68, author = {D. G. Luenberger}, journal = {SIAM Journal on Applied Mathematics}, number = {5}, pages = {1090--1095}, publisher = {SIAM}, title = {{Quasi-Convex Programming}}, volume = {16}, year = {1968} }
@article{LugosiMehrabian18, author = {G. Lugosi and A. Mehrabian}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1808.08416}{arXiv:1808.08416}}}, title = {Multiplayer Bandits Without Observing Collision Information}, url = {https://arxiv.org/abs/1808.08416}, year = {2018} }
@article{Maghsudi16, author = {S. Maghsudi and E. Hossain}, doi = {10.1109/MWC.2016.7498076}, journal = {Wireless Communications}, number = {3}, pages = {64--73}, publisher = {IEEE}, title = {{Multi-Armed Bandits with application to 5G small cells}}, volume = {23}, year = {2016} }
@article{Melian15, author = {L. Meli{\'a}n-Guti{\'e}rrez and N. Modi and C. Moy and F. Bader and I. P{\'e}rez-{\'A}lvarez and S. Zazo}, journal = {IEEE Transactions on Cognitive Communications and Networking}, number = {3}, pages = {347--358}, publisher = {IEEE}, title = {{Hybrid UCB-HMM: A Machine Learning Strategy for Cognitive Radio in HF Band}}, volume = {1}, year = {2015} }
@article{Mitola99, author = {J. Mitola and G. Q. Maguire}, journal = {{Personal Communications}}, number = {4}, pages = {13--18}, publisher = {IEEE}, title = {{Cognitive Radio: making software radios more personal}}, volume = {6}, year = {1999} }
@article{Modi17QoS, author = {N. Modi and P. Mary and C. Moy}, journal = {{Transactions on Cognitive Communications and Networking}}, number = {1}, pages = {49--66}, publisher = {IEEE}, title = {{QoS driven Channel Selection Algorithm for Cognitive Radio Network: Multi-User Multi-Armed Bandit Approach}}, volume = {3}, year = {2017} }
@article{Nielsen11, author = {F. Nielsen}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1102.2684}{arXiv:1102.2684}}}, title = {{Chernoff Information of Exponential Families}}, url = {https://arxiv.org/abs/1102.2684}, year = {2011} }
@article{Patil2019stochastic, author = {V. Patil and G. Ghalme and V. Nair and Y. Narahari}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1905.11260}{arXiv:1905.11260}}}, title = {{Stochastic Multi-Armed Bandits with Arm-specific Fairness Guarantees}}, url = {https://arxiv.org/abs/1905.11260}, year = {2019} }
@article{Raza17, author = {U. Raza and P. Kulkarni and M. Sooriyabandara}, doi = {10.1109/COMST.2017.2652320}, journal = {Communications Surveys Tutorials}, number = {2}, pages = {855--873}, publisher = {IEEE}, title = {{Low Power Wide Area Networks (LPWAN): An Overview}}, volume = {19}, year = {2017} }
@article{Robbins52, author = {H. Robbins}, journal = {{Bulletin of the American Mathematical Society}}, number = {5}, pages = {527--535}, title = {{Some Aspects of the Sequential Design of Experiments}}, volume = {58}, year = {1952} }
@article{Roberts75, author = {L. G. Roberts}, journal = {SIGCOMM Computer Communication Review}, number = {2}, pages = {28--42}, publisher = {ACM}, title = {{ALOHA Packet System With and Without Slots and Capture}}, volume = {5}, year = {1975} }
@article{Seznec2018, adress = {Naha, Japan}, author = {J. Seznec and A. Locatelli and A. Carpentier and A. Lazaric and M. Valko}, journal = {International Conference on Artificial Intelligence and Statistics}, title = {{Rotting Bandits Are No Harder Than Stochastic Ones}}, url = {https://arxiv.org/abs/1811.11043}, year = {2019} }
@article{Singla17, author = {S. Adish and H. Hassani and A. Krause}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1702.04825}{arXiv:1702.04825}}}, title = {{Learning to Use Learners' Advice}}, url = {https://arxiv.org/abs/1702.04825}, year = {2017} }
@article{Slivkins2019, author = {A. Slivkins}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1904.07272v3}{arXiv:1904.07272v3}}}, month = {June}, title = {{Introduction to Multi-Armed Bandits}}, url = {https://arxiv.org/abs/1904.07272v3}, year = {2019} }
@article{Thompson33, author = {W. R. Thompson}, journal = {{Biometrika}}, title = {{On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples}}, volume = {25}, year = {1933} }
@article{TianWangWangSong2019, author = {{Z. Tian and J. Wang and J. Wang and J. Song}}, doi = {10.1109/LWC.2019.2908371}, issn = {2162-2337}, journal = {{IEEE Wireless Communications Letters}}, number = {}, pages = {1--4}, title = {{Distributed NOMA-Based Multi-Armed Bandit Approach for Channel Access in Cognitive Radio Networks}}, volume = {}, year = {2019} }
@article{Truzzi13, author = {F. S. Truzzi and V. F. da Silva and A. H. Reali Costa and F. Gagliardi Cozman}, journal = {{ENIAC}}, number = {1}, title = {{AdBandit: a New Algorithm for Multi-Armed Bandits}}, volume = {2013}, year = {2013} }
@article{Wald45, author = {A. Wald}, journal = {The Annals of Mathematical Statistics}, number = {3}, pages = {287--293}, publisher = {Institute of Mathematical Statistics}, title = {{Some Generalizations of the Theory of Cumulative Sums of Random Variables}}, volume = {16}, year = {1945} }
@article{Whittle1988, author = {P. Whittle}, journal = {Journal of Applied Probability}, number = {A}, pages = {287--298}, publisher = {Cambridge University Press}, title = {{Restless bandits: Activity allocation in a changing world}}, volume = {25}, year = {1988} }
@article{Wilhelmi19collaborative, author = {F. Wilhelmi and C. Cano and G. Neu and B. Bellalta and A. Jonsson and S. Barrachina-Mu{\~n}oz}, journal = {Ad Hoc Networks}, publisher = {Elsevier}, title = {Collaborative Spatial Reuse In Wireless Networks Via Selfish Multi-Armed Bandits}, year = {2019} }
@article{Wilhelmi19potential, author = {F. Wilhelmi and S. Barrachina-Mu{\~n}oz and B. Bellalta and C. Cano and A. Jonsson and G. Neu}, journal = {Journal of Network and Computer Applications}, pages = {26--42}, publisher = {Elsevier}, title = {Potential and Pitfalls of Multi-Armed Bandits for Decentralized Spatial Reuse in WLANs}, volume = {127}, year = {2019} }
@article{Wilks1938, author = {S. S. Wilks}, journal = {The Annals of Mathematical Statistics}, number = {1}, pages = {60--62}, publisher = {JSTOR}, title = {{The large-sample distribution of the likelihood ratio for testing composite hypotheses}}, volume = {9}, year = {1938} }
@article{Yaari77, author = {M. E. Yaari}, journal = {Econometrica}, number = {5}, pages = {1183--1186}, publisher = {Wiley, Econometric Society}, title = {{A Note on Separability and Quasiconcavity}}, volume = {45}, year = {1977} }
@article{Zhao07, author = {Q. Zhao and B. M. Sadler}, journal = {{Signal Processing magazine}}, number = {3}, pages = {79--89}, publisher = {IEEE}, title = {{A Survey of Dynamic Spectrum Access}}, volume = {24}, year = {2007} }
@article{Zhao10, author = {K. Liu and Q. Zhao}, journal = {Transaction on Signal Processing}, number = {11}, pages = {5667--5681}, publisher = {IEEE}, title = {{Distributed Learning in Multi-Armed Bandit with Multiple Players}}, volume = {58}, year = {2010} }
@article{akyildiz2006next, author = {I. F. Akyildiz and W.-Y. Lee and M. C. Vuran and S. Mohanty}, journal = {Computer Networks}, number = {13}, pages = {2127--2159}, publisher = {Elsevier}, title = {{NeXt Generation, Dynamic Spectrum Access, Cognitive Radio Wireless Networks: A Survey}}, volume = {50}, year = {2006} }
@article{audibert2010minimax, author = {J.-Y. Audibert and S. Bubeck}, journal = {{Journal of Machine Learning Research}}, pages = {2785--2836}, title = {{Regret Bounds And Minimax Policies Under Partial Monitoring}}, volume = {11}, year = {2010} }
@article{barnard1959control, author = {G.A. Barnard}, journal = {Journal of the Royal Statistical Society. Series B (Methodological)}, pages = {239--271}, publisher = {JSTOR}, title = {{Control charts and stochastic processes}}, year = {1959} }
@article{bouneffouf2019survey, author = {D. Bouneffouf and I. Rish}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1904.10040}{arXiv:1904.10040}, under review by IJCAI 2019 Survey}}, title = {{A Survey on Practical Applications of Multi-Armed and Contextual Bandits}}, url = {https://arxiv.org/abs/1904.10040}, year = {2019} }
@article{chernoff1952measure, author = {H. Chernoff and others}, journal = {The Annals of Mathematical Statistics}, number = {4}, pages = {493--507}, publisher = {Institute of Mathematical Statistics}, title = {A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on The Sum of Observations}, volume = {23}, year = {1952} }
@article{chernoff1981note, author = {H. Chernoff}, journal = {The Annals of Probability}, pages = {533--535}, publisher = {JSTOR}, title = {A Note on an Inequality Involving the Normal Distribution}, year = {1981} }
@article{gao2019batched, author = {Z. Gao and Y. Han and Z. Ren and Z. Zhou}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1904.01763}{arXiv:1904.01763}}}, title = {{Batched Multi-armed Bandits Problem}}, url = {https://arxiv.org/abs/1904.01763}, year = {2019} }
@article{garhwal2012survey, arxiv = {{arXiv preprint \href{https://arXiv.org/abs/1201.1964}{arXiv:1201.1964}}}, author = {A. Garhwal and P. P. Bhattacharya}, doi = {10.5121/ijngn.2011.3402}, journal = {{International Journal of Next-Generation Networks (IJNGN)}}, number = {4}, title = {{A Survey on Dynamic Spectrum Access Techniques for Cognitive Radio}}, url = {https://arxiv.org/abs/1201.1964}, volume = {3}, year = {2012} }
@article{hagerup1990guided, author = {T. Hagerup and C. R{\"u}b}, journal = {Information processing letters}, number = {6}, pages = {305--308}, publisher = {Elsevier}, title = {{A Guided Tour of Chernoff Bounds}}, volume = {33}, year = {1990} }
@article{hoeffding1963probability, author = {W. Hoeffding}, journal = {Journal of the American statistical association}, number = {301}, pages = {13--30}, publisher = {Taylor \& Francis Group}, title = {Probability Inequalities for Sums of Bounded Random Variables}, volume = {58}, year = {1963} }
@article{ipython, author = {F. P\'erez and B. E. Granger}, howpublished = {Online at: \texttt{\url{https://ipython.org}}}, journal = {Computing in Science and Engineering}, month = {May}, number = {3}, pages = {21--29}, publisher = {IEEE Computer Society}, title = {{IPython: a System for Interactive Scientific Computing}}, url = {https://ipython.org}, volume = {9}, year = {2007} }
@article{kolnogorov2019multi, author = {A. Kolnogorov and S. Garbar}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1902.00214}{arXiv:1902.00214}}}, title = {{Multi-Armed Bandit Problem and Batch UCB Rule}}, url = {https://arxiv.org/abs/1902.00214}, year = {2019} }
@article{kumar2016two, author = {R. Kumar and S. J. Darak and A. Yadav and A. K. Sharma and R. K. Tripathi}, journal = {Wireless Networks}, pages = {1--15}, publisher = {Springer}, title = {{Two-stage Decision Making Policy for Opportunistic Spectrum Access and Validation on USRP Testbed}}, year = {2016} }
@article{kumar2017channel, author = {R. Kumar and S. J. Darak and A. Yadav and A. K. Sharma and R. K. Tripathi}, journal = {Communications Letters}, number = {10}, pages = {2186--2189}, publisher = {IEEE}, title = {{Channel Selection for Secondary Users in Decentralized Network of Unknown Size}}, volume = {21}, year = {2017} }
@article{li2019useDoublingTrick, author = {H. Li and J. Luo and C. Liu}, journal = {IEEE Access}, publisher = {IEEE}, title = {{Selfish Bandit based Cognitive Anti-jamming Strategy for Aeronautic Swarm Network in Presence of Multiple Jammert}}, year = {2019} }
@article{marinho2012cognitive, author = {J. Marinho and E. Monteiro}, journal = {Wireless Networks}, number = {2}, pages = {147--164}, publisher = {Springer-Verlag New York, Inc.}, title = {{Cognitive Radio: Survey on Communication Protocols Spectrum Decision Issues and Future Research Directions}}, volume = {18}, year = {2012} }
@article{matplotlib, author = {J. D. Hunter}, doi = {10.1109/MCSE.2007.55}, journal = {Computing In Science \& Engineering}, number = {3}, pages = {90--95}, publisher = {IEEE Computer Society}, title = {{Matplotlib: a 2D Graphics Environment}}, volume = {9}, year = {2007} }
@article{numpy, author = {S. van der Walt and C. S. Colbert and G. Varoquaux}, doi = {10.1109/mcse.2011.37}, journal = {Computing in Science {\&} Engineering}, month = {March}, number = {2}, pages = {22--30}, publisher = {IEEE}, title = {{The NumPy Array: A Structure for Efficient Numerical Computation}}, volume = {13}, year = {2011} }
@article{perchet2016, author = {P. Vianney and R. Philippe and C. Sylvain and S. Erik}, doi = {10.1214/15-AOS1381}, journal = {The Annals of Statistics}, month = {04}, number = {2}, pages = {660--681}, publisher = {The Institute of Mathematical Statistics}, title = {Batched Bandit Problems}, url = {https://doi.org/10.1214/15-AOS1381}, volume = {44}, year = {2016} }
@article{siegmund1995using, author = {D. Siegmund and E.S. Venkatraman}, journal = {The Annals of Statistics}, pages = {255--271}, publisher = {JSTOR}, title = {{Using the Generalized Likelihood Ratio Statistic for Sequential Detection of a Change Point}}, year = {1995} }
@article{subhedar2011spectrum, author = {M. Subhedar and G. Birajdar}, journal = {International Journal of Next-Generation Networks}, number = {2}, pages = {37--51}, publisher = {Academy \& Industry Research Collaboration Center (AIRCC)}, title = {{Spectrum Sensing Techniques in Cognitive Radio Networks: a Survey}}, volume = {3}, year = {2011} }
@article{tao2019collaborative, author = {C. Tao and Q. Zhang and Y. Zhou}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1904.03293}{arXiv:1904.03293}}}, title = {{Collaborative Learning with Limited Interaction: Tight Bounds for Distributed Exploration in Multi-Armed Bandits}}, url = {https://arxiv.org/abs/1904.03293}, year = {2019} }
@article{wang2019distributed, author = {Y. Wang and J. Hu and X. Chen and L. Wang}, journal = {{arXiv preprint \href{https://arXiv.org/abs/1904.06309}{arXiv:1904.06309}}}, title = {{Distributed Bandit Learning: How Much Communication is Needed to Achieve (Near) Optimal Regret}}, url = {https://arxiv.org/abs/1904.06309}, year = {2019} }
@article{yucek2009survey, author = {T. Yucek and H. Arslan}, journal = {IEEE Communications Surveys \& Tutorials}, number = {1}, pages = {116--130}, publisher = {IEEE}, title = {{A Survey of Spectrum Sensing Algorithms for Cognitive Radio Applications}}, volume = {11}, year = {2009} }
@incollection{Jamieson17ABTest, author = {F. Yang and A. Ramdas and K. Jamieson and M. Wainwright}, booktitle = {Advances in Neural Information Processing Systems}, pages = {5957--5966}, publisher = {Curran Associates, Inc.}, title = {{A framework for Multi-A(rmed)/B(andit) Testing with Online FDR Control}}, year = {2017} }
@incollection{LiChapelle11, author = {O. Chapelle and L. Li}, booktitle = {Advances in Neural Information Processing Systems}, pages = {2249--2257}, publisher = {Curran Associates, Inc.}, title = {{An Empirical Evaluation of Thompson Sampling}}, year = {2011} }
@inproceedings{Abramson1970, acmid = {1478502}, address = {New York, NY, USA}, author = {N. Abramson}, booktitle = {Proceedings of the November 17-19, 1970, Fall Joint Computer Conference}, doi = {10.1145/1478462.1478502}, location = {Houston, Texas}, numpages = {5}, pages = {281--285}, publisher = {ACM}, series = {AFIPS '70 (Fall)}, title = {{The ALOHA System: Another Alternative for Computer Communications}}, year = {1970} }
@inproceedings{Agarwal09explore, author = {D. Agarwal and B. Chen and P. Elango}, booktitle = {International Conference on Data Mining}, organization = {IEEE}, title = {{Explore Exploit Schemes For Web Content Optimization}}, year = {2009} }
@inproceedings{Agarwal16, author = {A. Agarwal and H. Luo and B. Neyshabur and R. E. Schapire}, booktitle = {{Conference on Learning Theory}}, pages = {12--38}, publisher = {{PMLR}}, title = {{Corralling a Band of Bandit Algorithms}}, year = {2017} }
@inproceedings{AgrawalGoyal11, author = {S. Agrawal and N. Goyal}, booktitle = {{Conference On Learning Theory}}, pages = {36--65}, publisher = {{PMLR}}, title = {{Analysis of Thompson sampling for the Multi-Armed Bandit problem}}, year = {2012} }
@inproceedings{Alami17, author = {R. Alami and O.-A. Maillard and R. F{\'e}raud}, booktitle = {{Conference on Neural Information Processing Systems}}, title = {{Memory Bandits: Towards the Switching Bandit Problem Best Resolution}}, year = {2017} }
@inproceedings{Allesiardo15, author = {R. Allesiardo and R. F{\'e}raud}, booktitle = {{International Conference on Data Science and Advanced Analytics}}, organization = {IEEE}, pages = {1--7}, title = {{Exp3 with Drift Detection for the Switching Bandit Problem}}, year = {2015} }
@inproceedings{Anandkumar10, author = {A. Anandkumar and N. Michael and A. K. Tang}, booktitle = {{International Conference on Computer Communications}}, organization = {IEEE}, title = {{Opportunistic Spectrum Access with multiple users: Learning under competition}}, year = {2010} }
@inproceedings{Audibert07, address = {Sendai, Japan}, author = {J.-Y. Audibert and R. Munos and C. Szepesv{\'a}ri}, booktitle = {{Algorithmic Learning Theory}}, organization = {Springer}, pages = {150--165}, title = {{Tuning Bandit Algorithms in Stochastic Environments}}, year = {2007} }
@inproceedings{Audibert2009minimax, author = {J-Y. Audibert and S. Bubeck}, booktitle = {{Conference on Learning Theory}}, pages = {217--226}, publisher = {{PMLR}}, title = {{Minimax Policies for Adversarial and Stochastic Bandits}}, year = {2009} }
@inproceedings{Auer16, author = {P. Auer and C.-K. Chiang}, booktitle = {{Conference on Learning Theory}}, pages = {116--120}, publisher = {{PMLR}}, title = {{An Algorithm with Nearly Optimal Pseudo Regret for Both Stochastic and Adversarial Bandits}}, year = {2016} }
@inproceedings{Avner16, author = {O. Avner and S. Mannor}, booktitle = {{International Conference on Computer Communications}}, organization = {IEEE}, title = {{Multi-User Lax Communications: a Multi-Armed Bandit Approach}}, year = {2016} }
@inproceedings{Azari18, address = {Abu Dhabi, UAE}, author = {A. Azari and C. Cavdar}, booktitle = {Global Communications Conference}, month = {December}, publisher = {IEEE}, title = {{Self-organized Low-power IoT Networks: A Distributed Learning Approach}}, year = {2018} }
@inproceedings{Besbes14stochastic, author = {O. Besbes and Y. Gur and A. Zeevi}, booktitle = {Advances in Neural Information Processing Systems}, pages = {199--207}, title = {{Stochastic Multi-Armed Bandit Problem with Non-Stationary Rewards}}, year = {2014} }
@inproceedings{Besson2018ALT, address = {Lanzarote, Spain}, author = {L. Besson and E. Kaufmann}, booktitle = {{Algorithmic Learning Theory}}, editor = {{M. Mohri and K. Sridharan}}, howpublished = {Online at: \texttt{\url{https://hal.archives-ouvertes.fr/hal-01629733}}. Code at: \texttt{\url{https://SMPyBandits.GitHub.io/MultiPlayers.html}}}, title = {{Multi-Player Bandits Revisited}}, url = {https://hal.archives-ouvertes.fr/hal-01629733}, year = {2018} }
@inproceedings{Besson2018WCNC, address = {Barcelona, Spain}, author = {L. Besson and E. Kaufmann and C. Moy}, booktitle = {{Wireless Communications and Networking Conference}}, howpublished = {Online at: \texttt{\url{https://hal.archives-ouvertes.fr/hal-01705292}}. Code at: \texttt{\url{https://SMPyBandits.GitHub.io/Aggregation.html}}}, publisher = {IEEE}, title = {{Aggregation of Multi-Armed Bandits Learning Algorithms for Opportunistic Spectrum Access}}, url = {https://hal.archives-ouvertes.fr/hal-01705292}, year = {2018} }
@inproceedings{Besson2019WCNC, address = {Marrakech, Morocco}, author = {L. Besson and R. Bonnefoi and C. Moy}, booktitle = {{Wireless Communications and Networking Conference}}, month = {April}, note = {Following a Demonstration presented at International Conference on Telecommunications (ICT) 2018}, publisher = {IEEE}, title = {{GNU Radio Implementation of MALIN: ``Multi-Armed bandits Learning for Internet-of-things Networks''}}, url = {https://hal.archives-ouvertes.fr/hal-02006825}, year = {2019} }
@inproceedings{Bistritz18, author = {I. Bistritz and A. Leshem}, booktitle = {Advances in Neural Information Processing Systems}, pages = {7222--7232}, title = {{Distributed Multi-Player Bandits: a Game Of Thrones Approach}}, year = {2018} }
@inproceedings{Bonnefoi16, author = {R. Bonnefoi and C. Moy and J. Palicot}, booktitle = {{International Conference on Communications, Control, and Computing Technologies for Smart Grids}}, doi = {10.1109/SmartGridComm.2016.7778766}, pages = {230--236}, title = {{Advanced metering infrastructure backhaul reliability improvement with Cognitive Radio}}, year = {2016} }
@inproceedings{Bonnefoi17, address = {Lisboa, Portugal}, author = {R. Bonnefoi and L. Besson and C. Moy and E. Kaufmann and J. Palicot}, booktitle = {{12th EAI Conference on Cognitive Radio Oriented Wireless Network and Communication}}, title = {{Multi-Armed Bandit Learning in IoT Networks: Learning Helps Even in Non-Stationary Settings}}, year = {2017} }
@inproceedings{Bonnefoi2019WCNC, address = {Marrakech, Morocco}, author = {R. Bonnefoi and L. Besson and J. C. Manco-Vasquez and C. Moy}, booktitle = {{MOTIoN Workshop}}, month = {April}, publisher = {IEEE}, title = {{Upper-Confidence Bound for Channel Selection in LPWA Networks with Retransmissions}}, url = {https://hal.archives-ouvertes.fr/hal-02049824}, year = {2019} }
@inproceedings{BubeckSlivkins12, author = {S. Bubeck and A. Slivkins}, booktitle = {{Conference on Learning Theory}}, pages = {42--1}, publisher = {{PMLR}}, title = {{The Best Of Both Worlds Stochastic And Adversarial Bandits}}, year = {2012} }
@inproceedings{Cano18, address = {New York, NY, USA}, author = {C. Cano and G. Neu}, booktitle = {Proceedings of the 2018 Workshop on Network Meets AI \& ML}, doi = {10.1145/3229543.3229551}, isbn = {978-1-4503-5911-5}, location = {Budapest, Hungary}, numpages = {7}, pages = {41--47}, publisher = {ACM}, series = {NetAI'18}, title = {Wireless Optimisation via Convex Bandits: Unlicensed LTE/WiFi Coexistence}, year = {2018} }
@inproceedings{CaoZhenKvetonXie18, address = {Okinawa, Japan}, author = {Y. Cao and W. Zheng and B. Kveton and Y. Xie}, booktitle = {International Conference on Artificial Intelligence and Statistics}, title = {{Nearly Optimal Adaptive Procedure for Piecewise-Stationary Bandit: a Change-Point Detection Approach}}, year = {2019} }
@inproceedings{ChenLeeLuoWei2019, arxiv = {{arXiv preprint \href{https://arXiv.org/abs/1902.00980}{arXiv:1902.00980}}}, author = {Y. Chen and C. Lee and H. Luo and C. Wei}, booktitle = {{Conference on Learning Theory}}, editor = {{A. Beygelzimer and D. Hsu}}, pages = {1--30}, publisher = {{PMLR}}, title = {{A New Algorithm for Non-stationary Contextual Bandits: Efficient, Optimal, and Parameter-free}}, url = {https://arxiv.org/abs/1902.00980}, volume = {99}, year = {2019} }
@inproceedings{Combes17, author = {R. Combes and S. Magureanu and A. Proutiere}, booktitle = {{Advances in Neural Information Processing Systems}}, pages = {1761--1769}, title = {{Minimal Exploration in Structured Stochastic Bandits}}, year = {2017} }
@inproceedings{Corless96, author = {R. Corless and G. Gonnet and D. Hare and D. Jeffrey and D. Knuth}, booktitle = {Advances in Computational Mathematics}, pages = {329--359}, publisher = {}, title = {{On the Lambert $\mathcal{W}$ Function}}, year = {1996} }
@inproceedings{Degenne16, author = {R. Degenne and V. Perchet}, booktitle = {{International Conference on Machine Learning}}, pages = {1587--1595}, title = {{Anytime Optimal Algorithms In Stochastic Multi Armed Bandits}}, year = {2016} }
@inproceedings{Garivier11KL, author = {A. Garivier and O. Capp{\'e}}, booktitle = {{Conference on Learning Theory}}, pages = {359--376}, publisher = {{PMLR}}, title = {{The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond}}, year = {2011} }
@inproceedings{Garivier11UCBDiscount, author = {A. Garivier and E. Moulines}, booktitle = {{Algorithmic Learning Theory}}, pages = {174--188}, publisher = {{PMLR}}, title = {{On Upper-Confidence Bound Policies For Switching Bandit Problems}}, year = {2011} }
@inproceedings{Garivier16BAI, author = {A. Garivier and E. Kaufmann}, booktitle = {{PMLR}}, series = {{Conference on Learning Theory}}, title = {{Optimal Best Arm Identification with Fixed Confidence}}, volume = {49}, year = {2016} }
@inproceedings{GarivierETC2016, author = {A. Garivier and E. Kaufmann and T. Lattimore}, booktitle = {{PMLR}}, fakeaddress = {Barcelona, Spain}, series = {Advances in Neural Information Processing Systems}, title = {{On Explore-Then-Commit Strategies}}, volume = {29}, year = {2016} }
@inproceedings{Gupta11thompson, author = {N. Gupta and O. Granmo-Christoffer and A. Agrawala}, booktitle = {International Conference on Machine Learning and Applications Workshops}, organization = {IEEE}, pages = {484--489}, title = {{Thompson Sampling for Dynamic Multi Armed Bandits}}, year = {2011} }
@inproceedings{Hartland06, author = {C. Hartland and S. Gelly and N. Baskiotis and O. Teytaud and M. Sebag}, booktitle = {NeurIPS 2006 Workshop, Online Trading Between Exploration And Exploitation}, title = {{Multi-Armed Bandit, Dynamic Environments and Meta-Bandits}}, year = {2006} }
@inproceedings{Honda10, author = {J. Honda and A. Takemura}, booktitle = {{Conference on Learning Theory}}, pages = {67--79}, publisher = {{PMLR}}, title = {{An Asymptotically Optimal Bandit Algorithm for Bounded Support Models}}, year = {2010} }
@inproceedings{JoshiKumar2018, author = {H. Joshi and R. Kumar and A. Yadav and S. J. Darak}, booktitle = {2018 IEEE Wireless Communications and Networking Conference (WCNC)}, doi = {10.1109/WCNC.2018.8377315}, issn = {1558-2612}, number = {}, pages = {1--6}, title = {{Distributed Algorithm for Dynamic Spectrum Access in Infrastructure-Less Cognitive Radio Network}}, volume = {}, year = {2018} }
@inproceedings{Jouini09, author = {W. Jouini and D. Ernst and C. Moy and J. Palicot}, booktitle = {{International Conference Signals, Circuits and Systems}}, organization = {IEEE}, title = {{Multi-Armed Bandit Based Policies for Cognitive Radio's Decision Making Issues}}, year = {2009} }
@inproceedings{Jouini10, author = {W. Jouini and D. Ernst and C. Moy and J. Palicot}, booktitle = {{International Conference on Communications}}, doi = {10.1109/ICC.2010.5502014}, organization = {IEEE}, pages = {1--5}, title = {{Upper Confidence Bound Based Decision Making Strategies and Dynamic Spectrum Access}}, year = {2010} }
@inproceedings{Kalathil12, author = {D. Kalathil and N. Nayyar and R. Jain}, booktitle = {{Conference on Decision and Control}}, publisher = {IEEE}, title = {{Decentralized Learning for Multi-Player Multi-Armed Bandits}}, year = {2012} }
@inproceedings{Kaufmann12BUCB, author = {E. Kaufmann and O. Capp{\'e} and A. Garivier}, booktitle = {{International Conference on Artificial Intelligence and Statistics}}, pages = {592--600}, title = {{On Bayesian Upper Confidence Bounds for Bandit Problems}}, year = {2012} }
@inproceedings{Kaufmann12Thompson, author = {E. Kaufmann and N. Korda and R. Munos}, booktitle = {{Algorithmic Learning Theory}}, pages = {199--213}, publisher = {{PMLR}}, title = {{Thompson Sampling: an Asymptotically Optimal Finite-Time Analysis}}, year = {2012} }
@inproceedings{Kaufmann14, author = {E. Kaufmann and O. Capp{\'e} and A. Garivier}, booktitle = {{Conference on Learning Theory}}, pages = {461--481}, publisher = {{PMLR}}, title = {{On the Complexity of A/B Testing}}, year = {2014} }
@inproceedings{KerkoucheAlami18, address = {Saint-Malo, France}, author = {R. Kerkouche and R. Alami and R. F{\'e}raud and N. Varsier and P. Maill{\'e}}, booktitle = {International Conference on Telecommunications}, organization = {{J. Palicot and R. Pyndiah}}, title = {{Node-based optimization of LoRa transmissions with Multi-Armed Bandit algorithms}}, year = {2018} }
@inproceedings{Kocsis06, author = {L. Kocsis and C. Szepesv{\'a}ri}, booktitle = {{2nd PASCAL Challenges Workshop}}, title = {{Discounted UCB}}, year = {2006} }
@inproceedings{Komiyama15, author = {J. Komiyama and J. Honda and H. Nakagawa}, booktitle = {{International Conference on Machine Learning}}, pages = {1152--1161}, publisher = {{PMLR}}, title = {{Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-Armed Bandit Problem with Multiple Plays}}, volume = {37}, year = {2015} }
@inproceedings{Koren17, author = {K. Tomer and L. Roi and M. Yishay}, booktitle = {{Conference on Learning Theory}}, pages = {1242--1268}, publisher = {{PMLR}}, title = {{Bandits with Movement Costs and Adaptive Pricing}}, volume = {65}, year = {2017} }
@inproceedings{KumarYadav2018, author = {R. Kumar and A. Yadav and S. J. Darak and M. K. Hanawal}, booktitle = {2018 16th International Symposium on Modeling and Optimization in Mobile, Ad-Hoc, and Wireless Networks (WiOpt)}, doi = {10.23919/WIOPT.2018.8362858}, issn = {}, number = {}, pages = {1--8}, title = {{Trekking Based Distributed Algorithm for Opportunistic Spectrum Access in Infrastructure-Less Network}}, volume = {}, year = {2018} }
@inproceedings{KvetonSzepesvari2019, arxiv = {{arXiv preprint \href{https://arXiv.org/abs/1902.10089}{arXiv:1902.10089}}}, author = {B. Kveton and C. Szepesvari and M. Ghavamzadeh and C. Boutilier}, booktitle = {28th International Joint Conference on Artificial Intelligence (IJCAI 2019)}, title = {{Perturbed-History Exploration in Stochastic Multi-Armed Bandits}}, url = {https://arxiv.org/abs/1902.10089}, year = {2019} }
@inproceedings{Kwon17, author = {J. Kwon and V. Perchet and C. Vernade}, booktitle = {{Conference on Learning Theory}}, pages = {1269--1270}, title = {{Sparse Stochastic Bandits}}, year = {2017} }
@inproceedings{Lagree16, author = {P. Lagr{\'e}e and C. Vernade and O. Capp{\'e}}, booktitle = {{Advances in Neural Information Processing Systems}}, pages = {1597--1605}, title = {{Multiple-Play Bandits in the Position-Based Model}}, year = {2016} }
@inproceedings{Lattimore16a, author = {T. Lattimore}, booktitle = {{Conference on Learning Theory}}, pages = {1214--1245}, publisher = {{PMLR}}, title = {{Regret Analysis Of The Finite Horizon Gittins Index Strategy For Multi Armed Bandits}}, year = {2016} }
@inproceedings{Lazaric09, author = {A. Lazaric and R. Munos}, booktitle = {Conference on Learning Theory}, title = {{Hybrid Stochastic-Adversarial On-Line Learning}}, year = {2009} }
@inproceedings{Li10, author = {L. Li and W. Chu and J. Langford and R. E. Schapire}, booktitle = {International Conference on World Wide Web}, organization = {ACM}, pages = {661--670}, title = {{A Contextual-Bandit Approach to Personalized News Article Recommendation}}, year = {2010} }
@inproceedings{Liauetal2017, author = {D. Liau and E. Price and Z. Song and G. Yang}, booktitle = {{International Conference on Artificial Intelligence and Statistics}}, title = {{Stochastic Multi-Armed Bandits in Constant Space}}, year = {2018} }
@inproceedings{Liu08, author = {K. Liu and Q. Zhao}, booktitle = {{Annual Communications Society Conference on Sensor, Mesh and Ad-Hoc Communications and Networks Workshops}}, publisher = {IEEE}, title = {{A Restless Bandit Formulation of Opportunistic Access: Indexablity and Index Policy}}, year = {2008} }
@inproceedings{LiuLeeShroff17, author = {F. Liu and J. Lee and N. Shroff}, booktitle = {{The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI 2018)}}, title = {{A Change-Detection based Framework for Piecewise-stationary Multi-Armed Bandit Problem}}, year = {2018} }
@inproceedings{Lopez2009spectral, author = {M. L{\'o}pez-Ben{\'\i}tez and F. Casadevall and A. Umbert and J. P{\'e}rez-Romero and R. Hachemani and J. Palicot and C. Moy}, booktitle = {2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications}, organization = {IEEE}, pages = {1--9}, title = {{Spectral Occupation Measurements and Blind Standard Recognition Sensor for Cognitive Radio Networks}}, year = {2009} }
@inproceedings{Louedec16, author = {J. Lou{\"e}dec and L. Rossi and M. Chevalier and A. Garivier and J. Mothe}, booktitle = {18{\`e}me Conf{\'e}rence francophone sur l'Apprentissage Automatique, 2016 (Marseille, France)}, title = {{Algorithme de bandit et obsolescence : un mod{\`e}le pour la recommandation}}, year = {2016} }
@inproceedings{Luo18, author = {H. Luo and C. Wei and A. Agarwal and J. Langford}, booktitle = {Proceedings of the 31st Conference On Learning Theory}, editor = {S. Bubeck and V. Perchet and P. Rigollet}, pages = {1739--1776}, pdf = {http://proceedings.mlr.press/v75/luo18a/luo18a.pdf}, publisher = {PMLR}, series = {Proceedings of Machine Learning Research}, title = {{Efficient Contextual Bandits in Non-stationary Worlds}}, url = {https://http://proceedings.mlr.press/v75/luo18a.html}, volume = {75}, year = {2018} }
@inproceedings{Maillard11, author = {O.-A. Maillard and R. Munos}, booktitle = {{International Conference on Artificial Intelligence and Statistics}}, pages = {570--578}, title = {{Adaptive Bandits: Towards the best history-dependent strategy}}, year = {2011} }
@inproceedings{Maillard2018GLR, author = {O.-A. Maillard}, booktitle = {Algorithmic Learning Theory}, title = {{Sequential change-point detection: Laplace concentration of scan statistics and non-asymptotic delay bounds}}, year = {2019} }
@inproceedings{MellorShapiro13, author = {J. Mellor and J. Shapiro}, booktitle = {{Artificial Intelligence and Statistics}}, pages = {442--450}, title = {{Thompson Sampling in Switching Environments with Bayesian Online Change Detection}}, year = {2013} }
@inproceedings{Menard17, author = {P. M{\'e}nard and A. Garivier}, booktitle = {{Algorithmic Learning Theory}}, pages = {223--237}, publisher = {{PMLR}}, title = {{A Minimax and Asymptotically Optimal Algorithm for Stochastic Bandits}}, volume = {76}, year = {2017} }
@inproceedings{Mitton16, address = {Graz, Austria}, author = {A. Maskooki and V. Toldov and L. Clavier and V. Loscr{\'i} and N. Mitton}, booktitle = {{International Conference on Embedded Wireless Systems and Networks (dependability competition)}}, month = {February}, title = {{Competition: Channel Exploration/Exploitation Based on a Thompson Sampling Approach in a Radio Cognitive Environment}}, year = {2016} }
@inproceedings{Mourtada17, address = {Tokyo, Japan}, author = {J. Mourtada and O.-A. Maillard}, booktitle = {{Algorithmic Learning Theory}}, pages = {1--23}, series = {Proceedings of Algorithmic Learning Theory}, title = {{Efficient Tracking of a Growing Number of Experts}}, volume = {76}, year = {2017} }
@inproceedings{MoyBesson2019, address = {Santorin, Greece}, author = {C. Moy and L. Besson}, booktitle = {{ISIoT workshop}}, hal_id = {hal-02144465}, hal_version = {v1}, month = {May}, pdf = {https://hal.inria.fr/hal-02144465/file/MB__ISIoT_2019.pdf}, title = {{Decentralized Spectrum Learning for IoT Wireless Networks Collision Mitigation}}, url = {https://sites.google.com/view/ISIoT2019/}, urlhal = {https://hal.inria.fr/hal-02144465}, year = {2019} }
@inproceedings{MoyWSR2014, address = {Karlsruhe, Germany}, author = {C. Moy}, booktitle = {{WSR'14}}, pages = {10}, title = {{Reinforcement Learning Real Experiments for Opportunistic Spectrum Access}}, url = {https://hal-supelec.archives-ouvertes.fr/hal-00994975}, year = {2014} }
@inproceedings{NaparstekCohen17, author = {O. {Naparstek} and K. {Cohen}}, booktitle = {GLOBECOM 2017 - 2017 IEEE Global Communications Conference}, doi = {10.1109/GLOCOM.2017.8254101}, number = {}, pages = {1--7}, title = {{Deep Multi-User Reinforcement Learning for Dynamic Spectrum Access in Multichannel Wireless Networks}}, volume = {}, year = {2017} }
@inproceedings{RobertSDR2014, address = {Schaumburg, United States}, author = {C. Robert and C. Moy and H. Zhang}, booktitle = {{SDR-WinnComm'14}}, pages = {8}, title = {{Opportunistic Spectrum Access Learning Proof of Concept}}, url = {https://hal-supelec.archives-ouvertes.fr/hal-00994940}, year = {2014} }
@inproceedings{Rosenski16, author = {J. Rosenski and O. Shamir and L. Szlak}, booktitle = {{International Conference on Machine Learning}}, pages = {155--163}, publisher = {{PMLR}}, title = {{Multi-Player Bandits -- A Musical Chairs Approach}}, year = {2016} }
@inproceedings{Sani12, author = {A. Sani and A. Lazaric and R. Munos}, booktitle = {Advances in Neural Information Processing Systems}, pages = {3275--3283}, title = {{Risk-Aversion In Multi-Armed Bandits}}, year = {2012} }
@inproceedings{SawantKumar2018, address = {Shanghai, China}, arxiv = {{arXiv preprint \href{https://arXiv.org/abs/1803.06810}{arXiv:1803.06810}}}, author = {S. Sawant and R. Kumar and M. K. Hanawal and S. J. Darak}, booktitle = {{16th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt)}}, doi = {10.23919/WIOPT.2018.8362853}, organization = {IEEE}, title = {{Learning to Coordinate in a Decentralized Cognitive Radio Network in Presence of Jammers}}, url = {https://arxiv.org/abs/1803.06810}, year = {2018} }
@inproceedings{Seldin17, author = {Y. Seldin and G. Lugosi}, booktitle = {{Conference on Learning Theory}}, pages = {1--17}, publisher = {{PMLR}}, title = {{An Improved Parametrization and Analysis of the EXP3++ Algorithm for Stochastic and Adversarial Bandits}}, volume = {65}, year = {{2017}} }
@inproceedings{Tekin11performance, author = {C. Tekin and M. Liu}, booktitle = {{International Conference on Game Theory for Networks}}, organization = {Springer Berlin Heidelberg}, pages = {321--336}, title = {{Performance and Convergence of Multi-User Online Learning}}, year = {2011} }
@inproceedings{Tekin12IEEE, author = {C. Tekin and M. Liu}, booktitle = {{Military Communications Conference}}, publisher = {IEEE}, title = {{Online Learning in Decentralized Multi-User Spectrum Access with Synchronized Explorations}}, year = {2012} }
@inproceedings{Tibrewal2019, arxiv = {{arXiv preprint \href{https://arXiv.org/abs/1901.03868}{arXiv:1901.03868}}}, author = {H. Tibrewal and S. Patchala and M. K. Hanawal and S. J. Darak}, booktitle = {{IEEE Conference on Computer Communications (INFOCOM 2019)}}, organization = {{IEEE}}, pages = {1693--1701}, title = {{Distributed Learning and Optimal Assignment in Multiplayer Heterogeneous Networks}}, url = {https://arxiv.org/abs/1901.03868}, year = {2019} }
@inproceedings{Toldov16, address = {Valencia, Spain}, author = {V. Toldov and L. Clavier and V. Loscr{\'i} and N. Mitton}, booktitle = {PIMRC}, doi = {10.1109/PIMRC.2016.7794785}, month = {September}, pages = {1--6}, title = {{A Thompson Sampling Approach to Channel Exploration Exploitation Problem in Multihop Cognitive Radio Networks}}, year = {2016} }
@inproceedings{WeiSrivastava18Abruptly, author = {L. Wei and V. Srivatsva}, booktitle = {American Control Conference}, organization = {IEEE}, pages = {6291--6296}, title = {{On Abruptly-Changing And Slowly-Varying Multi-Armed Bandit Problems}}, year = {2018} }
@inproceedings{WeiSrivastava18Distributed, author = {L. Wei and V. Srivastava}, booktitle = {Conference on Decision and Control}, organization = {IEEE}, pages = {5783--5788}, title = {{On Distributed Multi-player Multi-Armed Bandit Problems in Abruptly-Changing Environment}}, year = {2018} }
@inproceedings{Yang12, author = {X. Yang and A. Fapojuwo and E. Egbogah}, booktitle = {Vehicular Technology Conference}, doi = {10.1109/VTCFall.2012.6399192}, month = {September}, number = {}, pages = {1--5}, publisher = {IEEE}, title = {{Performance Analysis and Parameter Optimization of Random Access Backoff Algorithm in LTE}}, year = {2012} }
@inproceedings{YuMannor09, author = {J. Y. Yu and S. Mannor}, booktitle = {{International Conference on Machine Learning}}, organization = {ACM}, pages = {1177--1184}, title = {{Piecewise-Stationary Bandit Problems with Side Observations}}, year = {2009} }
@inproceedings{Zafaruddin2019, address = {{Cannes, France}}, arxiv = {{arXiv preprint \href{https://arXiv.org/abs/1902.06353}{arXiv:1902.06353}}}, author = {S. M. Zafaruddin and I. Bistritz and A. Leshem and D. Niyato}, booktitle = {{20th IEEE International Workshop on SignalProcessing Advances in Wireless Communications (SPAWC)}}, title = {{Distributed Learning for Channel Allocation Over a Shared Spectrum}}, url = {https://arxiv.org/abs/1902.06353}, year = {2019} }
@inproceedings{Zimmert2018, author = {J. Zimmert and Y. Seldin}, booktitle = {Proceedings of Machine Learning Research}, editor = {K. Chaudhuri and M. Sugiyama}, pages = {467--475}, pdf = {http://proceedings.mlr.press/v89/zimmert19a/zimmert19a.pdf}, publisher = {PMLR}, series = {Proceedings of Machine Learning Research}, title = {{An Optimal Algorithm for Stochastic and Adversarial Bandits}}, url = {http://proceedings.mlr.press/v89/zimmert19a.html}, volume = {89}, year = {2019} }
@inproceedings{audibert2010best, author = {J-Y. Audibert and S. Bubeck and R. Munos}, booktitle = {{Conference on Learning Theory}}, pages = {13}, publisher = {{PMLR}}, title = {{Best Arm Identification in Multi-Armed Bandits}}, year = {2010} }
@inproceedings{auer1995gambling, author = {P. Auer and N. Cesa-Bianchi and Y. Freund and R. Schapire}, booktitle = {Annual Symposium on Foundations of Computer Science}, organization = {IEEE}, pages = {322--331}, title = {{Gambling in a Rigged Casino: The Adversarial Multi-Armed Bandit Problem}}, year = {1995} }
@inproceedings{bande2019adversarial, author = {M. Bande and V. V. Veeravalli}, booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, organization = {IEEE}, pages = {4514--4518}, title = {{Adversarial Multi-User Bandits for Uncoordinated Spectrum Access}}, year = {2019} }
@inproceedings{darak2016bayesian, address = {Grenoble, France}, author = {S. J. Darak and A. Nafkha and C. Moy and J. Palicot}, booktitle = {{11th EAI Conference on Cognitive Radio Oriented Wireless Network and Communication}}, title = {{Is Bayesian Multi Armed Bandit Algorithm Superior? Proof of Concept for Opportunistic Spectrum Access in Decentralized Networks}}, year = {2016} }
@inproceedings{jupyter, author = {T. Kluyver and others}, booktitle = {Positioning and Power in Academic Publishing: Players, Agents and Agendas}, editor = {F. Loizides and B. Schmidt}, organization = {IOS Press}, pages = {87--90}, title = {{Jupyter Notebooks -- a publishing format for reproducible computational workflows}}, year = {2016} }
@inproceedings{modiDemo2016, address = {Grenoble, France}, author = {S. J. Darak and N. Modi and A. Nafkha and C. Moy}, booktitle = {{11th EAI Conference on Cognitive Radio Oriented Wireless Network and Communication}}, title = {{Spectrum Utilization and Reconfiguration Cost Comparison of Various Decision Making Policies for Opportunistic Spectrum Access Using Real Radio Signals}}, year = {2016} }
@inproceedings{patil2011survey, author = {K. Patil and R. Prasad and K. Skouby}, booktitle = {2011 International Conference on Devices and Communications (ICDeCom)}, organization = {IEEE}, pages = {1--5}, title = {{A Survey of Worldwide Spectrum Occupancy Measurement Campaigns for Cognitive Radio}}, year = {2011} }
@inproceedings{valenta2010survey, author = {V. Valenta and R. Mar{\v{s}}{\'a}lek and G. Baudoin and M. Villegas and M. Suarez and F. Robert}, booktitle = {{5th EAI Conference on Cognitive Radio Oriented Wireless Network and Communication}}, organization = {IEEE}, pages = {1--5}, title = {{Survey on spectrum utilization in Europe: Measurements, analyses and observations}}, year = {2010} }
@mastersthesis{Chiusano2018, author = {F. Chiusano}, school = {Politecnico Di Milano, AI \& R Lab, Laboratorio di Intelligenza Artificiale e Robotica del Politecnico di Milano}, title = {{Breakpoint Prediction for the Abruptly-Switching Non-Stationary Multi-Armed Bandit Problem}}, year = {2018} }
@misc{EiMathWorld, author = {E. W. Weisstein}, howpublished = {Online at \texttt{\url{https://mathworld.wolfram.com/ExponentialIntegral.html}}}, note = {From MathWorld -- A Wolfram Web Resource}, title = {{Exponential Integral}}, year = {2017} }
@misc{EiWikipedia, author = {Collective}, howpublished = {Online at \texttt{\url{https://en.wikipedia.org/wiki/Exponential_integral}}}, note = {From Wikipedia, The Free Encyclopedia}, title = {{Exponential Integral}}, year = {2017} }
@misc{GNURadioCompanionDocumentation, howpublished = {Online at \texttt{\url{https://wiki.gnuradio.org/index.php/GNURadioCompanion}}}, note = {Accessed: 2018-09-25}, title = {{GNU Radio Companion Documentation and Website}} }
@misc{GNURadioDocumentation, howpublished = {Online at \texttt{\url{https://www.gnuradio.org/about/}}}, note = {Accessed: 2018-09-25}, title = {{GNU Radio Documentation and Website}} }
@misc{JordanCourseStatBerkeley, author = {M. Jordan}, howpublished = {Online at \texttt{\url{https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/}}}, note = {Chapter 8 covers the exponential family}, title = {{Stat 260/CS 294}}, year = {2010} }
@misc{OctoclockProduct, howpublished = {Online at \texttt{\url{https://www.ettus.com/product/details/OctoClock-G}}}, note = {Accessed: 2018-09-25}, title = {{OctoClock Clock Distribution Module with GPSDO - Ettus Research}} }
@misc{SMPyBandits, author = {L. Besson}, note = {Code at \texttt{\url{https://GitHub.com/SMPyBandits/SMPyBandits/}}, documentation at \texttt{\url{https://SMPyBandits.GitHub.io/}}}, title = {{SMPyBandits: an Open-Source Research Framework for Single and Multi-Players Multi-Arms Bandits (MAB) Algorithms in Python}}, year = {2016--2019} }
@misc{Toledano2014EnglishSummary, author = {J. Toledano}, howpublished = {Online at \texttt{\url{https://www.economie.gouv.fr/files/files/PDF/french-spectrum-mission-executive-summary-2014-06-25.pdf}}}, month = {June}, pages = {1}, pdf = {https://www.economie.gouv.fr/files/files/PDF/french-spectrum-mission-executive-summary-2014-06-25.pdf}, title = {{Executive Summary: Dynamic Spectrum Management For Innovation And Growth}}, url = {https://www.ladocumentationfrancaise.fr/rapports-publics/144000381/index.shtml}, year = {2014} }
@misc{Toledano2014FrenchFull, author = {J. Toledano}, howpublished = {Online at \texttt{\url{https://www.economie.gouv.fr/files/files/PDF/rapport-gestion-dynamique-spectre-2014-06-30.pdf}}}, month = {June}, pages = {128}, pdf = {https://www.economie.gouv.fr/files/files/PDF/rapport-gestion-dynamique-spectre-2014-06-30.pdf}, title = {Une gestion dynamique du spectre pour l'innovation et la croissance}, url = {https://www.ladocumentationfrancaise.fr/rapports-publics/144000381/index.shtml}, year = {2014} }
@misc{TorLibbandit, author = {T. Lattimore}, howpublished = {Online at: \texttt{\url{https://github.com/tor/libbandit}}}, title = {{Library for Multi-Armed Bandit Algorithms}}, url = {https://github.com/tor/libbandit}, year = {2016} }
@misc{USRPDocumentation, author = {Ettus}, howpublished = {Online at \texttt{\url{https://files.ettus.com/manual/page_usrp2.html}}}, note = {Accessed: 2018-09-25}, title = {{USRP Hardware Driver and USRP Manual}} }
@misc{VishMABjl, author = {V. Raj}, howpublished = {Online at: \texttt{\url{https://github.com/v-i-s-h/MAB.jl}}}, title = {{A Julia Package for providing Multi Armed Bandit Experiments}}, url = {https://github.com/v-i-s-h/MAB.jl}, year = {2017} }
@misc{cython, author = {S. Behnel and R. Bradshaw and Dag S. Seljebotn and G. Ewing and W. Stein and G. Gellner and others}, howpublished = {Online at: \texttt{\url{https://cython.org}}}, title = {Cython: C-Extensions for Python}, url = {https://cython.org}, year = {2019} }
@misc{h5py, author = {A. Collette and others}, howpublished = {Online at: \texttt{\url{https://www.h5py.org}}}, title = {{h5py: HDF5 for Python}}, url = {https://www.h5py.org}, year = {2018} }
@misc{joblib, author = {G. Varoquaux}, howpublished = {Online at: \texttt{\url{https://joblib.readthedocs.io}}}, month = {March}, title = {{Joblib: running Python functions as pipeline jobs}}, url = {https://joblib.readthedocs.io}, year = {2017} }
@misc{numba, author = {Anaconda Inc. and others}, howpublished = {Online at: \texttt{\url{https://numba.pydata.org}}}, title = {{Numba, NumPy aware dynamic Python compiler using LLVM}}, url = {https://numba.pydata.org}, year = {2017} }
@misc{pymaBandits, author = {O. Cappé and A. Garivier and E. Kaufmann}, note = {Online at: \texttt{\url{https://mloss.org/software/view/415}}}, title = {{pymaBandits}}, url = {https://mloss.org/software/view/415}, year = {2012}, _howpublished = {Online at: \texttt{\url{https://mloss.org/software/view/415}}} }
@misc{python, author = {Python Software Foundation}, howpublished = {Online at: \texttt{\url{https://www.python.org}}}, month = {October}, title = {Python Language Reference, version 3.6}, url = {https://www.python.org}, year = {2017} }
@misc{scipy, author = {E. Jones and T. E. Oliphant and P. Peterson and others}, howpublished = {Online at: \texttt{\url{https://www.scipy.org}}}, title = {{SciPy: Open source scientific tools for Python}}, url = {https://www.scipy.org}, year = {2001} }
@misc{seaborn, author = {M. Waskom and others}, doi = {10.5281/zenodo.883859}, howpublished = {Online at: \texttt{\url{https://seaborn.pydata.org}}}, month = {September}, title = {Seaborn: statistical data visualization}, url = {https://seaborn.pydata.org}, year = {2017} }
@misc{sphinx, author = {G. Brandl and others}, howpublished = {Online at: \texttt{\url{https://sphinx-doc.org}}}, title = {Sphinx: Python documentation generator}, url = {https://sphinx-doc.org}, year = {2018}, _month = {February} }
@misc{virtualenv, author = {I. Bicking and PyPA and others}, howpublished = {Online at: \texttt{\url{https://virtualenv.pypa.io}}}, month = {November}, title = {{Virtualenv: a tool to create isolated Python environments}}, url = {https://virtualenv.pypa.io}, year = {2016} }
@phdthesis{Bodinier17, addendum = {Details about our TestBed can be found in the Appendix}, author = {Q. Bodinier}, school = {{CentraleSup{\'e}lec}}, title = {{Coexistence of Communication Systems Based on Enhanced Multi-Carrier Waveforms with Legacy OFDM Networks}}, url = {https://www.theses.fr/2017REN1S091}, year = {2017} }
@phdthesis{Jouini12PhD, author = {W. Jouini}, school = {{CentraleSup{\'e}lec, IETR, Rennes}}, title = {{Contribution to Learning and Decision Making under Uncertainty for Cognitive Radio}}, url = {https://www.theses.fr/2012SUPL0010}, year = {2017} }
@phdthesis{Kaufmann12PhD, author = {E. Kaufmann}, school = {{Telecom ParisTech}}, title = {{Analysis of Bayesian and Frequentist Strategies for Sequential Resource Allocation}}, url = {https://www.theses.fr/2014ENST0056}, year = {2014} }
@phdthesis{Modi17PhD, author = {N. Modi}, school = {{CentraleSup{\'e}lec, IETR, Rennes}}, title = {{Machine Learning and Statistical Decision Making for Green Radio}}, url = {https://www.theses.fr/2017SUPL0002}, year = {2017} }
@phdthesis{Toldov17, author = {V. Toldov}, school = {{Universit{\'e} Lille 1 Sciences et technologies}}, title = {{Adaptive MAC Layer for Interference Limited WSN}}, url = {https://www.theses.fr/2017LIL10002}, year = {2017} }
@phdthesis{valko2016bandits, abstract = {We investigate the structural properties of certain sequential decision-making problems with limited feedback (bandits) in order to bring the known algorithmic solutions closer to a practical use. In the first part, we put a special emphasis on structures that can be represented as graphs on actions, in the second part we study the large action spaces that can be of exponential size in the number of base actions or even infinite. We show how to take advantage of structures over the actions and (provably) learn faster.}, author = {M. Valko}, school = {{\'{E}}cole normale sup{\'{e}}rieure de Cachan}, title = {{Bandits on Graphs and Structures}}, type = {Habilitation thesis to supervise research}, year = {2016} }
@unpublished{Besson2018DoublingTricks, author = {L. Besson and E. Kaufmann}, howpublished = {Online at: \texttt{\url{https://hal.archives-ouvertes.fr/hal-01736357}}. Code at: \texttt{\url{https://SMPyBandits.GitHub.io/DoublingTrick.html}}}, month = {February}, note = {Preprint, \texttt{\url{https://hal.archives-ouvertes.fr/hal-01736357}}}, pdf = {hal.archives-ouvertes.fr/hal-01736357/document}, title = {{What Doubling Trick Can and Can't Do for Multi-Armed Bandits}}, url = {https://hal.archives-ouvertes.fr/hal-01736357}, year = {2018} }
@unpublished{Besson2018ICT, address = {Saint-Malo, France}, author = {L. Besson and R. Bonnefoi and C. Moy}, howpublished = {Online at: \texttt{\url{https://hal.archives-ouvertes.fr/hal-02013866}}}, month = {June}, note = {{Demonstration presented at International Conference on Telecommunications}}, title = {{Multi-Arm Bandit Algorithms for Internet of Things Networks: A TestBed Implementation and Demonstration that Learning Helps}}, url = {https://ict-2018.org/demos, youtu.be/HospLNQhcMk}, year = {2018} }
@unpublished{Besson2019GLRT, author = {L. Besson and E. Kaufmann}, howpublished = {Online at: \texttt{\url{https://hal.archives-ouvertes.fr/hal-02006471}}. Code at: \texttt{\url{https://SMPyBandits.GitHub.io/NonStationary.html}}}, month = {February}, note = {Preprint, \texttt{\url{https://hal.archives-ouvertes.fr/hal-02006471}}, arXiv preprint \href{https://arXiv.org/abs/1902.01575}{arXiv:1902.01575}}, title = {{Combining the Generalized Likelihood Ratio Test and kl-UCB for Non-Stationary Bandits}}, url = {https://hal.archives-ouvertes.fr/hal-02006471}, year = {2019} }
@unpublished{MoyBesson2019Annales, author = {C. Moy and L. Besson and G. Delbarre and L. Toutain}, hal_id = {hal-XXX}, hal_version = {v1}, journal = {{Special volume on Machine Learning for Intelligent Wireless Communications and Networking, Annals of Telecommunications}}, month = {July}, note = {{Submitted for a special volume on Machine Learning for Intelligent Wireless Communications and Networking, Annals of Telecommunications}}, pdf = {https://hal.inria.fr/hal-XXX/file/MBDT__Annales_2019.pdf}, publisher = {{Institut Mines-T{\'e}l{\'e}com and Springer}}, title = {{Decentralized Spectrum Learning for Radio Collision Mitigation in Ultra-Dense IoT Networks: LoRaWAN Case Study and Measurements}}, url = {https://hal.inria.fr/hal-XXX}, year = {2019} }
@unpublished{Mukherje2019, author = {S. Mukherje and O.-A. Maillard}, note = {Working paper or preprint}, title = {{Improved Changepoint Detection for Piecewise i.i.d Bandits}}, url = {https://subhojyoti.github.io/pdf/aistats_2019.pdf}, year = {2019} }
@article{RajKalyani17, author = {V. Raj and S. Kalyani}, journalb = {{arXiv preprint \href{https://arXiv.org/abs/1707.09727}{arXiv:1707.09727}}}, title = {{Taming Non-Stationary Bandits: a Bayesian Approach}}, url = {https://arxiv.org/abs/1707.09727}, year = {2017} }
@unpublished{SMPyBanditsJMLR, author = {L. Besson}, note = {Preprint, submitted to JMLR MLOSS, \texttt{\url{https://hal.archives-ouvertes.fr/hal-01840022}}}, pdf = {hal.archives-ouvertes.fr/hal-01840022/document}, title = {{SMPyBandits: an Experimental Framework for Single and Multi-Players Multi-Arms Bandits Algorithms in Python}}, url = {https://hal.archives-ouvertes.fr/hal-01840022}, year = {2018} }
@book{Basseville93, author = {M. Basseville and I. Nikiforov}, publisher = {{Prentice Hall Englewood Cliffs}}, title = {{Detection of Abrupt Changes: Theory And Application}}, volume = {104}, year = {1993} }
@book{Bona11, author = {M. B{\'o}na}, publisher = {World Scientific}, title = {{A Walk Through Combinatorics: an Introduction to Enumeration and Graph Theory}}, year = {2011} }
@book{Boucheron2013, author = {S. Boucheron and G. Lugosi and P. Massart}, publisher = {Oxford university press}, title = {{Concentration Inequalities: A Nonasymptotic Theory of Independence}}, year = {2013} }
@book{BoydVanderberghe04, author = {S. Boyd and L. Vandenberghe}, publisher = {{Cambridge Univ. Press}}, title = {{Convex Optimization}}, year = {2004} }
@book{CesaLugosi06, author = {N. Cesa-Bianchi and G. Lugosi}, publisher = {{Cambridge University Press}}, title = {{Prediction, Learning, and Games}}, year = {2006} }
@book{LattimoreBanditAlgorithmsBook, author = {T. Lattimore and C. Szepesv{\'a}ri}, note = {Draft of Wednesday 1st of May, 2019, \texttt{\url{https://tor-lattimore.com/downloads/book/book.pdf}}}, publisher = {Cambridge University Press}, title = {{Bandit Algorithms}}, url = {https://tor-lattimore.com/downloads/book/book.pdf}, year = {2019} }
@book{Norris98, author = {J. R. Norris}, publisher = {Cambridge University Press, Cambridge}, series = {Cambridge Series in Statistical and Probabilistic Mathematics}, title = {{Markov Chains}}, volume = {2}, year = {1998} }
@book{Siegmund13, author = {D. Siegmund}, publisher = {{Springer Science \& Business Media}}, title = {{Sequential analysis: tests and confidence intervals}}, year = {2013} }
@book{SuttonBarto2018, author = {R. S. Sutton and A. G. Barto}, publisher = {MIT press}, title = {{Reinforcement Learning: An introduction}}, year = {2018} }
@book{Tartakovsky15Book, author = {A. Tartakovsky and I. Nikiforov and M. Basseville}, publisher = {{CRC Press}}, title = {{Sequential Analysis. Hypothesis Testing and Changepoint Detection}}, year = {2015} }
This file was generated by bibtex2html 1.99.