| --- |
| license: apache-2.0 |
| tags: |
| - summarization |
| datasets: billsum |
| widget: |
| - text: 'The people of the State of California do enact as follows: SECTIONHEADER |
| Section 1170.02 is added to the Penal Code, to read: 1170.02. A prisoner is not |
| eligible for resentence or recall pursuant to subdivision (e) of Section 1170 |
| if he or she was convicted of first-degree murder if the victim was a peace officer, |
| as defined in Section 830.1, 830.2, 830.3, 830.31, 830.32, 830.33, 830.34, 830.35, |
| 830.36, 830.37, 830.4, 830.5, 830.6, 830.10, 830.11, or 830.12, who was killed |
| while engaged in the performance of his or her duties, and the individual knew, |
| or reasonably should have known, that the victim was a peace officer engaged in |
| the performance of his or her duties, or the victim was a peace officer or a former |
| peace officer under any of the above-enumerated sections, and was intentionally |
| killed in retaliation for the performance of his or her official duties. SECTIONHEADER |
| Section 3550 of the Penal Code is amended to read: 3550. Notwithstanding any other |
| law, except as provided in subdivision (b), if the head physician of an institution |
| in which a prisoner is incarcerated determines, as provided in this section, that |
| the prisoner is permanently medically incapacitated with a medical condition that |
| renders him or her permanently unable to perform activities of basic daily living, |
| and results in the prisoner requiring 24-hour care, and that incapacitation did |
| not exist at the time of sentencing, the prisoner shall be granted medical parole |
| if the Board of Parole Hearings determines that the conditions under which he |
| or she would be released would not reasonably pose a threat to public safety. |
| This section does not alter or diminish the rights conferred under the Victims |
| Bill of Rights Act of 2008 . Subdivision (a) does not apply to any of the following: |
| A prisoner sentenced to death or life in prison without possibility of parole. |
| A prisoner who is serving a sentence for which parole, pursuant to subdivision |
| (a), is prohibited by any initiative statute. A prisoner who was convicted of |
| first-degree murder if the victim was a peace officer, as defined in Section 830.1, |
| 830.2, 830.3, 830.31, 830.32, 830.33, 830.34, 830.35, 830.36, 830.37, 830.4, 830.5, |
| 830.6, 830.10, 830.11, or 830.12, who was killed while engaged in the performance |
| of his or her duties, and the individual knew, or reasonably should have known, |
| that the victim was a peace officer engaged in the performance of his or her duties, |
| or the victim was a peace officer or a former peace officer under any of the above-enumerated |
| sections, and was intentionally killed in retaliation for the performance of his |
| or her official duties. When a physician employed by the Department of Corrections |
| and Rehabilitation who is the primary care provider for a prisoner identifies |
| a prisoner that he or she believes meets the medical criteria for medical parole |
| specified in subdivision (a), the primary care physician shall recommend to the |
| head physician of the institution where the prisoner is located that the prisoner |
| be referred to the Board of Parole Hearings for consideration for medical parole. |
| Within 30 days of receiving that recommendation, if the head physician of the |
| institution concurs in the recommendation of the primary care physician, he or |
| she shall refer the matter to the Board of Parole Hearings using a standardized |
| form and format developed by the department, and if the head physician of the |
| institution does not concur in the recommendation, he or she shall provide the |
| primary care physician with a written explanation of the reasons for denying the |
| referral. Notwithstanding any other provisions of this section, the prisoner or |
| his or her family member or designee may independently request consideration for |
| medical parole by contacting the head physician at the prison or the department. |
| Within 30 days of receiving the request, the head physician of the institution |
| shall, in consultation with the prisoners primary care physician, make a determination |
| regarding whether the prisoner meets the criteria for medical parole as specified |
| in subdivision (a) and, if the head physician of the institution determines that |
| the prisoner satisfies the criteria set forth in subdivision (a), he or she shall |
| refer the matter to the Board of Parole Hearings using a standardized form and |
| format developed by the department. If the head physician of the institution does |
| not concur in the recommendation, he or she shall provide the prisoner or his |
| or her family member or designee with a written explanation of the reasons for |
| denying the application. The Department of Corrections and Rehabilitation shall |
| complete parole plans for inmates referred to the Board of Parole Hearings for |
| medical parole consideration. The parole plans shall include, but not be limited |
| to, the inmates plan for residency and medical care. Notwithstanding any other |
| law, medical parole hearings shall be conducted by two-person panels consisting |
| of at least one commissioner. In the event of a tie vote, the matter shall be |
| referred to the full board for a decision. Medical parole hearings may be heard |
| in absentia. Upon receiving a recommendation from the head physician of the institution |
| where a prisoner is located for the prisoner to be granted medical parole pursuant |
| to subdivision (c) or (d), the board, as specified in subdivision (f), shall make |
| an independent judgment regarding whether the conditions under which the inmate |
| would be released pose a reasonable threat to public safety, and make written |
| findings related thereto. Notwithstanding any other law, the board or the Division |
| of Adult Parole Operations shall have the authority to impose any reasonable conditions |
| on prisoners subject to medical parole supervision pursuant to subdivision (a), |
| including, but not limited to, the requirement that the parolee submit to electronic |
| monitoring. As a further condition of medical parole, pursuant to subdivision |
| (a), the parolee may be required to submit to an examination by a physician selected |
| by the board for the purpose of diagnosing the parolees current medical condition. |
| In the event such an examination takes place, a report of the examination and |
| diagnosis shall be submitted to the board by the examining physician. If the board |
| determines, based on that medical examination, that the persons medical condition |
| has improved to the extent that the person no longer qualifies for medical parole, |
| the board shall return the person to the custody of the department. Notwithstanding |
| any other law establishing maximum periods for parole, a prisoner sentenced to |
| a determinate term who is placed on medical parole supervision prior to the earliest |
| possible release date and who remains eligible for medical parole, shall remain |
| on medical parole, pursuant to subdivision (a), until that earliest possible release |
| date, at which time the parolee shall commence serving that period of parole provided |
| by, and under the provisions of, Chapter 8 of Title 1. Notwithstanding any other |
| law establishing maximum periods for parole, a prisoner sentenced to an indeterminate |
| term who is placed on medical parole supervision prior to the prisoners minimum |
| eligible parole date, and who remains eligible for medical parole, shall remain |
| on medical parole pursuant to subdivision (a) until that minimum eligible parole |
| date, at which time the parolee shall be eligible for parole consideration under |
| all other provisions of Chapter 8 of Title 1. The Department of Corrections and |
| Rehabilitation shall, at the time a prisoner is placed on medical parole supervision |
| pursuant to subdivision (a), ensure that the prisoner has applied for any federal |
| entitlement programs for which the prisoner is eligible, and has in his or her |
| possession a discharge medical summary, full medical records, parole medications, |
| and all property belonging to the prisoner that was under the control of the department. |
| Any additional records shall be sent to the prisoners forwarding address after |
| release to health care-related parole supervision. The provisions for medical |
| parole set forth in this title shall not affect an inmates eligibility for any |
| other form of parole or release provided by law. (1) Notwithstanding any other |
| law, the Department of Corrections and Rehabilitation shall give notice to the |
| county of commitment and the proposed county of release, if that county is different |
| than the county of commitment, of any medical parole hearing as described in subdivision |
| (f), and of any medical parole release as described in subdivision (g). Notice |
| shall be made at least 30 days, or as soon as feasible, prior to the time any |
| medical parole hearing or medical parole release is scheduled for an inmate receiving |
| medical parole consideration, regardless of whether the inmate is sentenced either |
| determinately or indeterminately.' |
| base_model: led-base-16384 |
| model-index: |
| - name: d0r1h/LEDBill |
| results: |
| - task: |
| type: summarization |
| name: Summarization |
| dataset: |
| name: billsum |
| type: billsum |
| config: default |
| split: test |
| metrics: |
| - type: rouge |
| value: 38.6502 |
| name: ROUGE-1 |
| verified: true |
| - type: rouge |
| value: 18.5458 |
| name: ROUGE-2 |
| verified: true |
| - type: rouge |
| value: 25.6561 |
| name: ROUGE-L |
| verified: true |
| - type: rouge |
| value: 33.1575 |
| name: ROUGE-LSUM |
| verified: true |
| - type: loss |
| value: 2.1305277347564697 |
| name: loss |
| verified: true |
| - type: gen_len |
| value: 288.372 |
| name: gen_len |
| verified: true |
| --- |
| |
| # Longformer Encoder-Decoder (LED) fine-tuned on Billsum |
|
|
|
|
|
|
| This model is a fine-tuned version of [led-base-16384](https://huggingface.co/allenai/led-base-16384) on the [billsum](https://huggingface.co/datasets/billsum) dataset. |
|
|
| As described in [Longformer: The Long-Document Transformer](https://arxiv.org/pdf/2004.05150.pdf) by Iz Beltagy, Matthew E. Peters, Arman Cohan, *led-base-16384* was initialized from [*bart-base*](https://huggingface.co/facebook/bart-base) since both models share the exact same architecture. To be able to process 16K tokens, *bart-base*'s position embedding matrix was simply copied 16 times. |
|
|
|
|
| ## How to use |
|
|
| ```Python |
| from transformers import AutoModelForSeq2SeqLM, AutoTokenizer |
| |
| device = "cuda" if torch.cuda.is_available() else "cpu" |
| |
| tokenizer = AutoTokenizer.from_pretrained("d0r1h/LEDBill") |
| model = AutoModelForSeq2SeqLM.from_pretrained("d0r1h/LEDBill", return_dict_in_generate=True).to(device) |
| |
| case = "......." |
| |
| input_ids = tokenizer(case, return_tensors="pt").input_ids.to(device) |
| global_attention_mask = torch.zeros_like(input_ids) |
| global_attention_mask[:, 0] = 1 |
| |
| sequences = model.generate(input_ids, |
| global_attention_mask=global_attention_mask).sequences |
| summary = tokenizer.batch_decode(sequences, |
| skip_special_tokens=True) |
| |
| ``` |
|
|
|
|
| ## Evaluation results |
|
|
| When the model is used for summarizing Billsum documents(10 sample), it achieves the following results: |
|
|
| | Model | rouge1-f | rouge1-p | rouge2-f | rouge2-p | rougeL-f | rougeL-p | |
| |:-----------:|:-----:|:-----:|:------:|:-----:|:------:|:-----:| |
| | LEDBill | **34** | **37** | **15** | **16** | **30** | **32** | |
| | led-base | 2 | 15 | 0 | 0 | 2 | 15 | |
|
|
| [This notebook](https://colab.research.google.com/drive/1iEEFbWeTGUSDesmxHIU2QDsPQM85Ka1K?usp=sharing) shows how *led* can effectively be used for downstream task such summarization. |
|
|
|
|
