[docker-dev] Buy electronics credit clone card

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

[docker-dev] CREDIT CLONE CARD WITH HIGH AND LOW BALANCE AMOUNT

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

[docker-dev] Buy low balance credit clone card / fast and reliable at all ATM machine

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

[docker-dev] Buy skimming unauthorized credit clone card

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

[docker-dev] How to use credit clone card/buy with options to use credit clone card

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

[docker-dev] Fine out the best balance of credit clone card/ low and high balance k

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

Re: gwt-maven-springboot-archetype updated ...

['Frank Hossfeld' via GWT Users]

I'll added some additional code to avoid adding the launcherDir as document 
root in production mode. 
New version should be soon available.

Craig Mitchell schrieb am Samstag, 18. Mai 2024 um 16:00:47 UTC+2:

> The issue seems to be the launcherDir directory doesn't exist.  I've 
> raised a an issue with the full stack trace:  
> https://github.com/NaluKit/gwt-maven-springboot-archetype/issues/13
>
> On Saturday 18 May 2024 at 11:06:28 pm UTC+10 Frank Hossfeld wrote:
>
>> please can you post the error message: Thanks
>>
>> Craig Mitchell schrieb am Samstag, 18. Mai 2024 um 13:11:40 UTC+2:
>>
>>> I spoke to soon.  Adding the EmbeddedServletContainerConfig fixes the 
>>> serialization policy when running locally, but if you do a build and try to 
>>> run.  Ie:  mvn clean package and then java -jar myserver/myapp.war, it 
>>> crashes.
>>>
>>> I'll investigate.  Any ideas/help most welcome.
>>>
>>> On Saturday 18 May 2024 at 8:42:09 pm UTC+10 Craig Mitchell wrote:
>>>
>>>> > *New version available ... This one fixes the 
>>>> serializationPolicyFilePath issue ... *
>>>>
>>>> Awesome, thanks!  I've closed 
>>>> https://github.com/NaluKit/gwt-maven-springboot-archetype/issues/7  
>>>>
>>>> On Thursday 16 May 2024 at 4:38:52 am UTC+10 Frank Hossfeld wrote:
>>>>
>>>>> Bott -> Boot ...  (spelling correction  arrrgh)
>>>>>
>>>>>

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit/1f0480da-8548-4422-97c8-be1cce4a3839n%40googlegroups.com.

[kwin] [Bug 487189] Pointer confinement temporarily lost when leaving fullscreen

[Frank Praznik]

https://bugs.kde.org/show_bug.cgi?id=487189

--- Comment #1 from Frank Praznik  ---
To note, this works as expected in GNOME, Weston, and Sway, so not likely to be
an SDL bug.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kwin] [Bug 487189] New: Pointer confinement temporarily lost when leaving fullscreen

[Frank Praznik]

https://bugs.kde.org/show_bug.cgi?id=487189

Bug ID: 487189
   Summary: Pointer confinement temporarily lost when leaving
fullscreen
Classification: Plasma
   Product: kwin
   Version: 6.0.4
  Platform: Other
OS: Linux
Status: REPORTED
  Severity: normal
  Priority: NOR
 Component: input
  Assignee: kwin-bugs-n...@kde.org
  Reporter: frank.praz...@gmail.com
  Target Milestone: ---

Created attachment 169595
  --> https://bugs.kde.org/attachment.cgi?id=169595=edit
WAYLAND_DEBUG=1 output

After leaving fullscreen mode with pointer confinement active on a window, the
pointer is no longer confined until it leaves and re-enters the window.

This can be replicated with testwm in the SDL 3 test suite.

STEPS TO REPRODUCE
1. Run "testwm --grab" (the pointer should be confined to the window)
2. With the pointer in the window, press alt+enter to enter fullscreen.
3. With the pointer positioned such that it will be where the non-fullscreen
window will be restored to, press alt+enter to leave fullscreen.
4. The pointer can leave the window, and won't be confined again until it
re-enters.

The pointer should remain confined to the window without an additional
leave+enter cycle after leaving fullscreen.

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: Fedora 40
KDE Plasma Version: 6.0.4

-- 
You are receiving this mail because:
You are watching all bug changes.

Re: gwt-maven-springboot-archetype updated ...

['Frank Hossfeld' via GWT Users]

please can you post the error message: Thanks

Craig Mitchell schrieb am Samstag, 18. Mai 2024 um 13:11:40 UTC+2:

> I spoke to soon.  Adding the EmbeddedServletContainerConfig fixes the 
> serialization policy when running locally, but if you do a build and try to 
> run.  Ie:  mvn clean package and then java -jar myserver/myapp.war, it 
> crashes.
>
> I'll investigate.  Any ideas/help most welcome.
>
> On Saturday 18 May 2024 at 8:42:09 pm UTC+10 Craig Mitchell wrote:
>
>> > *New version available ... This one fixes the 
>> serializationPolicyFilePath issue ... *
>>
>> Awesome, thanks!  I've closed 
>> https://github.com/NaluKit/gwt-maven-springboot-archetype/issues/7  
>>
>> On Thursday 16 May 2024 at 4:38:52 am UTC+10 Frank Hossfeld wrote:
>>
>>> Bott -> Boot ...  (spelling correction  arrrgh)
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit/6e0fb4be-406a-4b97-b8a4-7fcd71387bccn%40googlegroups.com.

[Bug 2059303] Re: [UBUNTU 20.04] SE-tooling: New IBM host-key subject locality (s390-tools)

[Frank Heimes]

Thank you Steffen once again!
I'm updating the tags accordingly ...

** Tags removed: verification-needed verification-needed-focal 
verification-needed-jammy verification-needed-mantic
** Tags added: verification-done verification-done-focal 
verification-done-jammy verification-done-mantic

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2059303

Title:
  [UBUNTU 20.04] SE-tooling: New IBM host-key subject locality
  (s390-tools)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2059303/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [PATCH 05/12] dmaengine: Add STM32 DMA3 support

[Frank Li]

On Fri, May 17, 2024 at 11:42:17AM +0200, Amelie Delaunay wrote:
> On 5/16/24 19:09, Frank Li wrote:
> > On Thu, May 16, 2024 at 05:25:58PM +0200, Amelie Delaunay wrote:
> > > On 5/15/24 20:56, Frank Li wrote:
> > > > On Tue, Apr 23, 2024 at 02:32:55PM +0200, Amelie Delaunay wrote:
> > > > > STM32 DMA3 driver supports the 3 hardware configurations of the STM32 
> > > > > DMA3
> > > > > controller:
> > ...
> > > > > + writel_relaxed(hwdesc->cdar, ddata->base + STM32_DMA3_CDAR(id));
> > > > > + writel_relaxed(hwdesc->cllr, ddata->base + STM32_DMA3_CLLR(id));
> > > > > +
> > > > > + /* Clear any pending interrupts */
> > > > > + csr = readl_relaxed(ddata->base + STM32_DMA3_CSR(id));
> > > > > + if (csr & CSR_ALL_F)
> > > > > + writel_relaxed(csr, ddata->base + STM32_DMA3_CFCR(id));
> > > > > +
> > > > > + stm32_dma3_chan_dump_reg(chan);
> > > > > +
> > > > > + ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(id));
> > > > > + writel_relaxed(ccr | CCR_EN, ddata->base + STM32_DMA3_CCR(id));
> > > > 
> > > > This one should use writel instead of writel_relaxed because it need
> > > > dma_wmb() as barrier for preious write complete.
> > > > 
> > > > Frank
> > > > 
> > > 
> > > ddata->base is Device memory type thanks to ioremap() use, so it is 
> > > strongly
> > > ordered and non-cacheable.
> > > DMA3 is outside CPU cluster, its registers are accessible through AHB bus.
> > > dma_wmb() (in case of writel instead of writel_relaxed) is useless in that
> > > case: it won't ensure the propagation on the bus is complete, and it will
> > > have impacts on the system.
> > > That's why CCR register is written once,  then it is read before CCR_EN is
> > > set and being written again, with _relaxed(), because registers are 
> > > behind a
> > > bus, and ioremapped with Device memory type which ensures it is strongly
> > > ordered and non-cacheable.
> > 
> > regardless memory map, writel_relaxed() just make sure io write and read is
> > orderred, not necessary order with other memory access. only readl and
> > writel make sure order with other memory read/write.
> > 
> > 1. Write src_addr to descriptor
> > 2. dma_wmb()
> > 3. Write "ready" to descriptor
> > 4. enable channel or doorbell by write a register.
> > 
> > if 4 use writel_relaxe(). because 3 write to DDR, which difference place of
> > mmio, 4 may happen before 3.  Your can refer axi order model.
> > 
> > 4 have to use ONE writel(), to make sure 3 already write to DDR.
> > 
> > You need use at least one writel() to make sure all nornmal memory finish.
> > 
> 
> +writel_relaxed(chan->swdesc->ccr, ddata->base + STM32_DMA3_CCR(id));
> +writel_relaxed(hwdesc->ctr1, ddata->base + STM32_DMA3_CTR1(id));
> +writel_relaxed(hwdesc->ctr2, ddata->base + STM32_DMA3_CTR2(id));
> +writel_relaxed(hwdesc->cbr1, ddata->base + STM32_DMA3_CBR1(id));
> +writel_relaxed(hwdesc->csar, ddata->base + STM32_DMA3_CSAR(id));
> +writel_relaxed(hwdesc->cdar, ddata->base + STM32_DMA3_CDAR(id));
> +writel_relaxed(hwdesc->cllr, ddata->base + STM32_DMA3_CLLR(id));
> 
> These writel_relaxed() are from descriptors to DMA3 registers (descriptors
> being prepared "a long time ago" during _prep_).

You can't depend on "a long time ago" during _prep_. If later your driver
run at fast CPU. The execute time will be short.

All dma_map_sg and dma_alloc_coherence ... need at least one writel() to
make sure previous write actually reach to DDR. 

Some data may not really reach DDR, when DMA already start transfer

Please ref linux kernel document: 
Documentation/memory-barriers.txt, line 1948.

In your issue_pending(), call this function to enable channel. So need
at least one writel().

> As I said previously, DMA3 registers are outside CPU cluster, accessible
> through AHB bus, and ddata->base to address registers is ioremapped as
> Device memory type, non-cacheable and strongly ordered.
> 
> arch/arm/include/asm/io.h:
> /*
> * ioremap() and friends.
> *
> * ioremap() takes a resource address, and size.  Due to the ARM memory
> * types, it is important to use the correct ioremap() function as each
> * mapping has specific properties.
> *
> * FunctionMemory type CacheabilityCache hint
&g

[Bug 2050017] Re: [24.04 FEAT] [SEC2339] HSM protected signing support for Apache httpd for openSSL 3.0 with PKCS #11 provider

[Frank Heimes]

The IBM team agreed upon the proposal to let's go with the PPA solution
for now, until upstream accepted (and reconsider in this case).

(So I think I'm updating the status of this ticket to 'Opinion'.)

** Changed in: ubuntu-z-systems
   Status: Triaged => Opinion

** Changed in: apache2 (Ubuntu)
   Status: Triaged => Opinion

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2050017

Title:
  [24.04 FEAT] [SEC2339] HSM protected signing support for Apache httpd
  for openSSL 3.0 with PKCS #11 provider

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2050017/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[kmail2] [Bug 483779] KMail hangs when invoking "Configure KMail..."

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=483779

Peer Frank  changed:

   What|Removed |Added

 CC||peer.fr...@web.de

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kmail2] [Bug 483779] KMail hangs when invoking "Configure KMail..."

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=483779

Peer Frank  changed:

   What|Removed |Added

 CC||peer.fr...@web.de

-- 
You are receiving this mail because:
You are watching all bug changes.

[Bug 2059303] Re: [UBUNTU 20.04] SE-tooling: New IBM host-key subject locality (s390-tools)

[Frank Heimes]

** Changed in: ubuntu-z-systems
   Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2059303

Title:
  [UBUNTU 20.04] SE-tooling: New IBM host-key subject locality
  (s390-tools)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2059303/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [PATCH] target/riscv: zvbb implies zvkb

[Frank Chang]

Reviewed-by: Frank Chang 

On Thu, May 16, 2024 at 8:34 PM Jerry Zhang Jian 
wrote:

> - According to RISC-V crypto spec, Zvkb extension is a proper subset of
> the Zvbb extension.
>
> - Reference:
> https://github.com/riscv/riscv-crypto/blob/1769c2609bf4535632e0c0fd715778f212bb272e/doc/vector/riscv-crypto-vector-zvkb.adoc?plain=1#L10
>
> Signed-off-by: Jerry Zhang Jian 
> ---
>  target/riscv/tcg/tcg-cpu.c | 4 
>  1 file changed, 4 insertions(+)
>
> diff --git a/target/riscv/tcg/tcg-cpu.c b/target/riscv/tcg/tcg-cpu.c
> index 40054a391a..f1a1306ab2 100644
> --- a/target/riscv/tcg/tcg-cpu.c
> +++ b/target/riscv/tcg/tcg-cpu.c
> @@ -658,6 +658,10 @@ void riscv_cpu_validate_set_extensions(RISCVCPU *cpu,
> Error **errp)
>  cpu_cfg_ext_auto_update(cpu, CPU_CFG_OFFSET(ext_zvbc), true);
>  }
>
> +if (cpu->cfg.ext_zvbb) {
> +cpu_cfg_ext_auto_update(cpu, CPU_CFG_OFFSET(ext_zvkb), true);
> +}
> +
>  /*
>   * In principle Zve*x would also suffice here, were they supported
>   * in qemu
> --
> 2.42.0
>
>

Re: [RBW] ISO: Toyo Atlantis 56-58cm/26"

[Frank Brose]

No need for correction. 56 were 26 inch , the 58 were 700c. I've had both 
and preferred the 56 so I bought another one after I sold the 58. Just sayin
Frank 

On Thursday, May 16, 2024 at 5:48:31 PM UTC-5 NYCbikeguy wrote:

> Hey Jerry,
>
> Correct me if I'm wrong, but I believe the Toyo Atlantis 26inchers were 
> limited to 56cm frames and under. 58 and above were 700c.
>
> On Thursday, May 16, 2024 at 5:55:32 PM UTC-4 Matthew Williams wrote:
>
>> Atlantis 55 for sale 
>> <https://groups.google.com/g/rbw-owners-bunch/c/7Fm5e3JPKBM/m/NYuNyvQIAwAJ>
>> groups.google.com 
>> <https://groups.google.com/g/rbw-owners-bunch/c/7Fm5e3JPKBM/m/NYuNyvQIAwAJ>
>> [image: groups_32dp.png] 
>> <https://groups.google.com/g/rbw-owners-bunch/c/7Fm5e3JPKBM/m/NYuNyvQIAwAJ> 
>> <https://groups.google.com/g/rbw-owners-bunch/c/7Fm5e3JPKBM/m/NYuNyvQIAwAJ>
>>
>>
>> On May 16, 2024, at 2:54 PM, Jerry Lynn  wrote:
>>
>> Seeking a Toyo Atlantis 56-58cm/26" frameset please. Looking to hand over 
>> my 55cm/93 XO-1 to my wife - it's a bit small for me anyway - so would love 
>> to build a 26" Riv for myself (I currently have several 26" wheelsets).
>>
>> Jerry Lynn
>> Imperial Beach CA
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "RBW Owners Bunch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to rbw-owners-bun...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/rbw-owners-bunch/2d8b3623-8b39-4a35-9227-7951a8224b98n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/rbw-owners-bunch/2d8b3623-8b39-4a35-9227-7951a8224b98n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "RBW 
Owners Bunch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to rbw-owners-bunch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/rbw-owners-bunch/5af767cd-7618-4696-b7ba-816cf349c3f5n%40googlegroups.com.

[valgrind] [Bug 411203] valgrind should have a mean to pass environment variable to target program

[Frank Ch. Eigler]

https://bugs.kde.org/show_bug.cgi?id=411203

Frank Ch. Eigler  changed:

   What|Removed |Added

 CC||f...@redhat.com

--- Comment #5 from Frank Ch. Eigler  ---
Another case where this sort of feature would be useful relates to debuginfod. 
We may need a way of passing $DEBUGINFOD_URLS only to valgrind or only to the
target process (depending on what it is we're trying to debug).

Consider a command line option scheme kind of like docker's or /bin/env's:

valgrind -e ENV1= -e ENV2=foo   /bin/target

which could mean "drop ENV1 from, and override ENV2=foo in, valgrind's
environment"

-- 
You are receiving this mail because:
You are watching all bug changes.

Re: [PATCH 05/12] dmaengine: Add STM32 DMA3 support

[Frank Li]

On Thu, May 16, 2024 at 05:25:58PM +0200, Amelie Delaunay wrote:
> On 5/15/24 20:56, Frank Li wrote:
> > On Tue, Apr 23, 2024 at 02:32:55PM +0200, Amelie Delaunay wrote:
> > > STM32 DMA3 driver supports the 3 hardware configurations of the STM32 DMA3
> > > controller:
...
> > > + writel_relaxed(hwdesc->cdar, ddata->base + STM32_DMA3_CDAR(id));
> > > + writel_relaxed(hwdesc->cllr, ddata->base + STM32_DMA3_CLLR(id));
> > > +
> > > + /* Clear any pending interrupts */
> > > + csr = readl_relaxed(ddata->base + STM32_DMA3_CSR(id));
> > > + if (csr & CSR_ALL_F)
> > > + writel_relaxed(csr, ddata->base + STM32_DMA3_CFCR(id));
> > > +
> > > + stm32_dma3_chan_dump_reg(chan);
> > > +
> > > + ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(id));
> > > + writel_relaxed(ccr | CCR_EN, ddata->base + STM32_DMA3_CCR(id));
> > 
> > This one should use writel instead of writel_relaxed because it need
> > dma_wmb() as barrier for preious write complete.
> > 
> > Frank
> > 
> 
> ddata->base is Device memory type thanks to ioremap() use, so it is strongly
> ordered and non-cacheable.
> DMA3 is outside CPU cluster, its registers are accessible through AHB bus.
> dma_wmb() (in case of writel instead of writel_relaxed) is useless in that
> case: it won't ensure the propagation on the bus is complete, and it will
> have impacts on the system.
> That's why CCR register is written once,  then it is read before CCR_EN is
> set and being written again, with _relaxed(), because registers are behind a
> bus, and ioremapped with Device memory type which ensures it is strongly
> ordered and non-cacheable.

regardless memory map, writel_relaxed() just make sure io write and read is
orderred, not necessary order with other memory access. only readl and
writel make sure order with other memory read/write.

1. Write src_addr to descriptor
2. dma_wmb()
3. Write "ready" to descriptor
4. enable channel or doorbell by write a register.

if 4 use writel_relaxe(). because 3 write to DDR, which difference place of
mmio, 4 may happen before 3.  Your can refer axi order model.

4 have to use ONE writel(), to make sure 3 already write to DDR.

You need use at least one writel() to make sure all nornmal memory finish.

> 
> > > +
> > > + chan->dma_status = DMA_IN_PROGRESS;
> > > +
> > > + dev_dbg(chan2dev(chan), "vchan %pK: started\n", >vchan);
> > > +}
> > > +
> > > +static int stm32_dma3_chan_suspend(struct stm32_dma3_chan *chan, bool 
> > > susp)
> > > +{
> > > + struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
> > > + u32 csr, ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)) & 
> > > ~CCR_EN;
> > > + int ret = 0;
> > > +
> > > + if (susp)
> > > + ccr |= CCR_SUSP;
> > > + else
> > > + ccr &= ~CCR_SUSP;
> > > +
> > > + writel_relaxed(ccr, ddata->base + STM32_DMA3_CCR(chan->id));
> > > +
> > > + if (susp) {
> > > + ret = readl_relaxed_poll_timeout_atomic(ddata->base + 
> > > STM32_DMA3_CSR(chan->id), csr,
> > > + csr & CSR_SUSPF, 1, 10);
> > > + if (!ret)
> > > + writel_relaxed(CFCR_SUSPF, ddata->base + 
> > > STM32_DMA3_CFCR(chan->id));
> > > +
> > > + stm32_dma3_chan_dump_reg(chan);
> > > + }
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +static void stm32_dma3_chan_reset(struct stm32_dma3_chan *chan)
> > > +{
> > > + struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
> > > + u32 ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)) & 
> > > ~CCR_EN;
> > > +
> > > + writel_relaxed(ccr |= CCR_RESET, ddata->base + 
> > > STM32_DMA3_CCR(chan->id));
> > > +}
> > > +
> > > +static int stm32_dma3_chan_stop(struct stm32_dma3_chan *chan)
> > > +{
> > > + struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
> > > + u32 ccr;
> > > + int ret = 0;
> > > +
> > > + chan->dma_status = DMA_COMPLETE;
> > > +
> > > + /* Disable interrupts */
> > > + ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id));
> > > + writel_relaxed(ccr & ~(CCR_ALLIE | CCR_EN), ddata->base + 
> > > STM32_DMA3_CCR(chan->id));
> > > +
> > > + if (!(ccr & CCR_SUSP) &&am

[ceph-users] Re: Please discuss about Slow Peering

[Frank Schilder]

This is a long shot: if you are using octopus, you might be hit by this 
pglog-dup problem: https://docs.clyso.com/blog/osds-with-unlimited-ram-growth/. 
They don't mention slow peering explicitly in the blog, but its also a 
consequence because the up+acting OSDs need to go through the PG_log during 
peering.

We are also using octopus and I'm not sure if we have ever seen slow ops caused 
by peering alone. It usually happens when a disk cannot handle load under 
peering. We have, unfortunately, disks that show random latency spikes 
(firmware update pending). You can try to monitor OPS latencies for your drives 
when peering and look for something that sticks out. People on this list were 
reporting quite bad results for certain infamous NVMe brands. If you state your 
model numbers, someone else might recognize it.

Best regards,
=
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14


From: 서민우 
Sent: Thursday, May 16, 2024 7:39 AM
To: ceph-users@ceph.io
Subject: [ceph-users] Please discuss about Slow Peering

Env:
- OS: Ubuntu 20.04
- Ceph Version: Octopus 15.0.0.1
- OSD Disk: 2.9TB NVMe
- BlockStorage (Replication 3)

Symptom:
- Peering when OSD's node up is very slow. Peering speed varies from PG to
PG, and some PG may even take 10 seconds. But, there is no log for 10
seconds.
- I checked the effect of client VM's. Actually, Slow queries of mysql
occur at the same time.

There are Ceph OSD logs of both Best and Worst.

Best Peering Case (0.5 Seconds)
2024-04-11T15:32:44.693+0900 7f108b522700  1 osd.7 pg_epoch: 27368 pg[6.8]
state: transitioning to Primary
2024-04-11T15:32:45.165+0900 7f108f52a700  1 osd.7 pg_epoch: 27371 pg[6.8]
state: Peering, affected_by_map, going to Reset
2024-04-11T15:32:45.165+0900 7f108f52a700  1 osd.7 pg_epoch: 27371 pg[6.8]
start_peering_interval up [7,6,11] -> [6,11], acting [7,6,11] -> [6,11],
acting_primary 7 -> 6, up_primary 7 -> 6, role 0 -> -1, features acting
2024-04-11T15:32:45.165+0900 7f108f52a700  1 osd.7 pg_epoch: 27377 pg[6.8]
state: transitioning to Primary
2024-04-11T15:32:45.165+0900 7f108f52a700  1 osd.7 pg_epoch: 27377 pg[6.8]
start_peering_interval up [6,11] -> [7,6,11], acting [6,11] -> [7,6,11],
acting_primary 6 -> 7, up_primary 6 -> 7, role -1 -> 0, features acting

Worst Peering Case (11.6 Seconds)
2024-04-11T15:32:45.169+0900 7f108b522700  1 osd.7 pg_epoch: 27377 pg[30.20]
state: transitioning to Stray
2024-04-11T15:32:45.169+0900 7f108b522700  1 osd.7 pg_epoch: 27377 pg[30.20]
start_peering_interval up [0,1] -> [0,7,1], acting [0,1] -> [0,7,1],
acting_primary 0 -> 0, up_primary 0 -> 0, role -1 -> 1, features acting
2024-04-11T15:32:46.173+0900 7f108b522700  1 osd.7 pg_epoch: 27378 pg[30.20]
state: transitioning to Stray
2024-04-11T15:32:46.173+0900 7f108b522700  1 osd.7 pg_epoch: 27378 pg[30.20]
start_peering_interval up [0,7,1] -> [0,7,1], acting [0,7,1] -> [0,1],
acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> -1, features acting
2024-04-11T15:32:57.794+0900 7f108b522700  1 osd.7 pg_epoch: 27390 pg[30.20]
state: transitioning to Stray
2024-04-11T15:32:57.794+0900 7f108b522700  1 osd.7 pg_epoch: 27390 pg[30.20]
start_peering_interval up [0,7,1] -> [0,7,1], acting [0,1] -> [0,7,1],
acting_primary 0 -> 0, up_primary 0 -> 0, role -1 -> 1, features acting

*I wish to know about*
- Why some PG's take 10 seconds until Peering finishes.
- Why Ceph log is quiet during peering.
- Is this symptom intended in Ceph.

*And please give some advice,*
- Is there any way to improve peering speed?
- Or, Is there a way to not affect the client when peering occurs?

P.S
- I checked the symptoms in the following environments.
-> Octopus Version, Reef Version, Cephadm, Ceph-Ansible
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

Re: [DBWG] why can't i change role object? -- resolved

[Frank Habicht]

Hi all,

My problem was that (as a test whether i can modify the object) i tried 
to change the first line/attribute: "role".


this is a "lookup key" per [1] and can not be changed.
Changing other attributes works.

Thanks Keessun for your help!

We agreed that documentation might need improvements.


Have a good time  - and let's have the cables fixed soon

Frank
(no hat)


[1]
$ whois -h whois.afrinic.net -- -t role
[Querying whois.afrinic.net]
[whois.afrinic.net]
% This is the AfriNIC Whois server.
% The AFRINIC whois database is subject to  the following terms of Use. 
See https://afrinic.net/whois/terms


role:   [mandatory]  [single] [lookup key]
address:[mandatory]  [multiple]   [ ]
phone:  [optional]   [multiple]   [ ]
fax-no: [optional]   [multiple]   [ ]
e-mail: [mandatory]  [multiple]   [lookup key]
org:[optional]   [multiple]   [inverse key]
admin-c:[mandatory]  [multiple]   [inverse key]
tech-c: [mandatory]  [multiple]   [inverse key]
nic-hdl:[mandatory]  [single] [primary/lookup key]
remarks:[optional]   [multiple]   [ ]
notify: [optional]   [multiple]   [inverse key]
abuse-mailbox:  [optional]   [multiple]   [inverse key]
mnt-by: [optional]   [multiple]   [inverse key]
changed:[mandatory]  [multiple]   [ ]
source: [mandatory]  [single] [ ]




On 16/05/2024 07:20, Sylvain Baya wrote:

Dear DBWG,
Hope this email finds you in good health!

Please see my comments below, inline.
Thanks.

Le ven. 10 mai 2024 à 12:52 PM, Frank Habicht <mailto:ge...@geier.ne.tz>> a écrit :


Hi all,


Hi Frank,
Thanks for keeping the WG as active as you can, brother :'-(


I hope/trust that's not a question for "Support"...


...sure!
But, someone should explain how Policy
Manual have been used in building the
business rules behind implementation :-)


I tried to update/modify a role object, and got this (sorry for the
line
breaks):


What's the difference with other objects?

PII (Person Identifiable Information) risk?


***Error:   Person/Role name cannot be changed automatically. Please
create


...time to *automate* the process?


              another Person/Role object and modify any references
to the old
              object, then delete the old object



:-/ so the object's history would be lost?



I can't see a reason why 



...i don't also agree that:
'delete' right
is easily grantable than
'update' right

:-/ ...i would have understood a kind of
intent implementation of a "protect *new created* objects in order to 
avoid an  accidental deletion" policy...


Such as:

"A WhoisDB object can only be deleted
after a *reasonable* amount of time; at least :
  $new_delete_prevent_time seconds"

->Protected object! please try again after
  $new_delete_prevent_time seconds;
in case you *really* need to delete it ;-)


updates to role projects should not be allowed.
I even think that's the main purpose

So, am I missing something?


Thanks,
Frank



PS: AfriNIC staff:
I got "Internal software error" when i wanted to delete
SNHT5-AFRINIC
less than 1 minute after creating it


...interesting behaviour :-\

< $new_delete_prevent_time?


wanted to delete it with the line
delete:         remove
added

according to
delete: 
under section "2.2.4 Deleting an object"
in
https://afrinic.net/press/197-database-afrinic-database-reference-manual- 
<https://afrinic.net/press/197-database-afrinic-database-reference-manual->

deleting without the comment worked.


Noted!
...by the way, have you first reproduced
your initial testing steps/constraints[*]?
:-)
__
[*]: (i) create an object; (ii) delete within a minute; (iii) ...


Can that section in that manual be reviewed?


For the *remastering* activity concerning
  the Whois Manual & documentation, it
should be done openly & collaboratively;
imho!

...it would be interesting to contribute on
it under a public/locally [.] accessible git
repository or a self hosted [.] wiki engine.
__
[.]: idea  or dot afrinic?



If google sent me to an outdated document, can we remove it?


...or simply versioned and archived?

Please see here [1] for appropriate facts.
__
[1]: AFRINIC's Factsheets
<https://afrinic.net/our-factsheets/ <https://afrinic.net/our-factsheets/>>




Thanks,
Frank



Shalom,
--sb.

--
Best Regards !

baya.sylvain [AT cmNOG DOT cm]
| "cmNOG's Structure"<https://www.cmnog.cm/dokuwiki/Structure 
<https://www.cmnog.cm/dokuwiki/Structure>> | "Douala-IX's IXP 
Manager"<https://ixpm.douala-ix.net/statistics/ 
<https://ixpm.douala-ix.net/statistics/>> |
"cmNOG's Surveys"<https://survey2.cmnog.cm <https://survey2.cmnog.cm>> | 

Re: [gentoo-user] Graphics configuration for a Ryzen 7 7700X chip and water cooling.

[Frank Steinmetzger]

Am Wed, May 15, 2024 at 07:08:11PM +0100 schrieb Michael:
> Hi Alan,
> 
> On Wednesday, 15 May 2024 15:23:47 BST Alan Mackenzie wrote:
> > Hello, Gentoo.
> > […]
> > So I'm looking at getting an AMD Ryzen 7 7700X processor, and using its
> > inbuilt graphics rather than buying a distinct graphics card.
> > 
> […]
> > As a somewhat tangential question, would it be worthwhile getting water
> > cooling in my new machine?  In particular, to reduce the noise it gives
> > off while building large packages such as clang and rust?  Or is water
> > cooling only sensible for really heavy users such as gamers?
> > 
> > Thanks for the upcoming answers!
> 
> WC will be quieter and more expensive than an after market air cooler.

Are you sure about the noise? First there is the water pump and second, 
the heat from the air cycle needs to get somewhere, which is donw with fans.
So unless you get a big radiator with several fans, you just relocate the 
fan noise inside the case.

I have a 10 years old i5 with a TDP of I think 84 W. On that sits a normal 
(not even high-performance) tower cooler with a single 120 mm fan. At full 
load the CPU draws around 50 W, maybe even less unless you do prime95. So my 
cooler is basically overkill. But this allows the fan to never leave the 
minimum RPM range of ~500…600 1/min and is unaudible even at full load.
However …

> You could invest the money toward more RAM, (more/bigger) case fans, a 
> better PSU, monitor, speakers, a new car, etc.  :-)
> 
> https://www.techreviewer.com/tech-specs/amd-7700x-tdp/
> 
> Cranking up 16 threads to 5.4 GHz will produce some heat, but compiles will 
> complete sooner too.

… the 7000X are hotheads, because they operate way above the efficiency 
sweetspot just to get the longest bar in benchmark diagrams. If you reduce 
the power target¹ in the BIOS, you lose a few percent in performance, but 
get a disproportionately bigger reduction in energy consumption.

¹ The TDP of a 7700X is 105 W. The maximum permanent power draw is TDP * 1.4 
(ish, can’t remember the exact details right now). So if you reduce the 
target to 84 W, you draw a little over 100 W. That’s easy-peasy for a mormal 
120 mm tower cooler. One additional advantage of an air cooler is that it 
also blows air over your mainboard and its power stages. That’s something 
you don’t get with a water loop and need an extra case fan for—IF you keep 
the CPU on high load all the time which causes more heat buildup in the VRMs.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The perfect diet: no breakfast in the morning,
in return forego pudding at lunch and then go to bed without dinner.


signature.asc
Description: PGP signature

Re: [PATCH v2 03/15] hw/riscv: add RISC-V IOMMU base emulation

[Frank Chang]

On Mon, May 13, 2024 at 8:37 PM Daniel Henrique Barboza <
dbarb...@ventanamicro.com> wrote:

> Hi Frank,
>
>
> On 5/8/24 08:15, Daniel Henrique Barboza wrote:
> > Hi Frank,
> >
> > I'll reply with that I've done so far. Still missing some stuff:
> >
> > On 5/2/24 08:37, Frank Chang wrote:
> >> Hi Daniel,
> >>
> >> Daniel Henrique Barboza  於 2024年3月8日 週五
> 上午12:04寫道：
> >>>
> >>> From: Tomasz Jeznach 
> >>>
> >>> The RISC-V IOMMU specification is now ratified as-per the RISC-V
> >>> international process. The latest frozen specifcation can be found
> >>> at:
> >>>
> >>>
> https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0/riscv-iommu.pdf
> >>>
> >>> Add the foundation of the device emulation for RISC-V IOMMU, which
> >>> includes an IOMMU that has no capabilities but MSI interrupt support
> and
> >>> fault queue interfaces. We'll add add more features incrementally in
> the
> >>> next patches.
> >>>
> >>> Co-developed-by: Sebastien Boeuf 
> >>> Signed-off-by: Sebastien Boeuf 
> >>> Signed-off-by: Tomasz Jeznach 
> >>> Signed-off-by: Daniel Henrique Barboza 
> >>> ---
> >>>   hw/riscv/Kconfig |4 +
>
> (...)
>
> >>> +
> >>> +s->iommus.le_next = NULL;
> >>> +s->iommus.le_prev = NULL;
> >>> +QLIST_INIT(>spaces);
> >>> +qemu_cond_init(>core_cond);
> >>> +qemu_mutex_init(>core_lock);
> >>> +qemu_spin_init(>regs_lock);
> >>> +qemu_thread_create(>core_proc, "riscv-iommu-core",
> >>> +riscv_iommu_core_proc, s, QEMU_THREAD_JOINABLE);
> >>
> >> In our experience, using QEMU thread increases the latency of command
> >> queue processing,
> >> which leads to the potential IOMMU fence timeout in the Linux driver
> >> when using IOMMU with KVM,
> >> e.g. booting the guest Linux.
> >>
> >> Is it possible to remove the thread from the IOMMU just like ARM, AMD,
> >> and Intel IOMMU models?
> >
> > Interesting. We've been using this emulation internally in Ventana, with
> > KVM and VFIO, and didn't experience this issue. Drew is on CC and can
> talk
> > more about it.
> >
> > That said, I don't mind this change, assuming it's feasible to make it
> for this
> > first version.  I'll need to check it how other IOMMUs are doing it.
>
>
> I removed the threading and it seems to be working fine without it. I'll
> commit this
> change for v3.
>
> >
> >
> >
> >>
> >>> +}
> >>> +
> >
> > (...)
> >
> >>> +
> >>> +static AddressSpace *riscv_iommu_find_as(PCIBus *bus, void *opaque,
> int devfn)
> >>> +{
> >>> +RISCVIOMMUState *s = (RISCVIOMMUState *) opaque;
> >>> +PCIDevice *pdev = pci_find_device(bus, pci_bus_num(bus), devfn);
> >>> +AddressSpace *as = NULL;
> >>> +
> >>> +if (pdev && pci_is_iommu(pdev)) {
> >>> +return s->target_as;
> >>> +}
> >>> +
> >>> +/* Find first registered IOMMU device */
> >>> +while (s->iommus.le_prev) {
> >>> +s = *(s->iommus.le_prev);
> >>> +}
> >>> +
> >>> +/* Find first matching IOMMU */
> >>> +while (s != NULL && as == NULL) {
> >>> +as = riscv_iommu_space(s, PCI_BUILD_BDF(pci_bus_num(bus),
> devfn));
> >>
> >> For pci_bus_num(),
> >> riscv_iommu_find_as() can be called at the very early stage
> >> where software has no chance to enumerate the bus numbers.
>
> I investigated and this doesn't seem to be a problem. This function is
> called at the
> last step of the realize() steps of both riscv_iommu_pci_realize() and
> riscv_iommu_sys_realize(), and by that time the pci_bus_num() is already
> assigned.
> Other iommus use pci_bus_num() into their own get_address_space()
> callbacks like
> this too.
>

Hi Daniel,

IIUC, pci_bus_num() by default is assigned to pcibus_num():

static int pcibus_num(PCIBus *bus)
{
if (pci_bus_is_root(bus)) {
return 0; /* pci host bridge */
}
return bus->parent_dev->config[PCI_SECONDARY_BUS];
}

If the bus is not the root bus, it tries to read the bus' parent device's
secondary bus number (PCI_SECONDARY_BUS

Re: [PATCH 05/12] dmaengine: Add STM32 DMA3 support

[Frank Li]

; + _ctr1 |= CTR1_DAP;
> +
> + _ctr2 |= FIELD_PREP(CTR2_REQSEL, chan->dt_config.req_line) & 
> ~CTR2_SWREQ;
> + if (FIELD_GET(STM32_DMA3_DT_BREQ, tr_conf))
> + _ctr2 |= CTR2_BREQ;
> + if (dir == DMA_DEV_TO_MEM && FIELD_GET(STM32_DMA3_DT_PFREQ, tr_conf))
> + _ctr2 |= CTR2_PFREQ;
> + tcem = FIELD_GET(STM32_DMA3_DT_TCEM, tr_conf);
> + _ctr2 |= FIELD_PREP(CTR2_TCEM, tcem);
> +
> + /* Store TCEM to know on which event TC flag occurred */
> + chan->tcem = tcem;
> + /* Store direction for residue computation */
> + chan->dma_config.direction = dir;
> +
> + switch (dir) {
> + case DMA_MEM_TO_DEV:
> + /* Set destination (device) data width and burst */
> + ddw = min_t(u32, ddw, stm32_dma3_get_max_dw(chan->max_burst, 
> dap_max_dw,
> + len, dst_addr));
> + dbl_max = min_t(u32, dbl_max, stm32_dma3_get_max_burst(len, 
> ddw, chan->max_burst));
> +
> + /* Set source (memory) data width and burst */
> + sdw = stm32_dma3_get_max_dw(chan->max_burst, sap_max_dw, len, 
> src_addr);
> + sbl_max = stm32_dma3_get_max_burst(len, sdw, chan->max_burst);
> +
> + _ctr1 |= FIELD_PREP(CTR1_SDW_LOG2, ilog2(sdw));
> + _ctr1 |= FIELD_PREP(CTR1_SBL_1, sbl_max - 1);
> + _ctr1 |= FIELD_PREP(CTR1_DDW_LOG2, ilog2(ddw));
> + _ctr1 |= FIELD_PREP(CTR1_DBL_1, dbl_max - 1);
> +
> + if (ddw != sdw) {
> + _ctr1 |= FIELD_PREP(CTR1_PAM, CTR1_PAM_PACK_UNPACK);
> + /* Should never reach this case as ddw is clamped down 
> */
> + if (len & (ddw - 1)) {
> + dev_err(chan2dev(chan),
> + "Packing mode is enabled and len is not 
> multiple of ddw");
> + return -EINVAL;
> + }
> + }
> +
> + /* dst = dev */
> + _ctr2 |= CTR2_DREQ;
> +
> + break;
> +
> + case DMA_DEV_TO_MEM:
> + /* Set source (device) data width and burst */
> + sdw = min_t(u32, sdw, stm32_dma3_get_max_dw(chan->max_burst, 
> sap_max_dw,
> + len, src_addr));
> + sbl_max = min_t(u32, sbl_max, stm32_dma3_get_max_burst(len, 
> sdw, chan->max_burst));
> +
> + /* Set destination (memory) data width and burst */
> + ddw = stm32_dma3_get_max_dw(chan->max_burst, dap_max_dw, len, 
> dst_addr);
> + dbl_max = stm32_dma3_get_max_burst(len, ddw, chan->max_burst);
> +
> + _ctr1 |= FIELD_PREP(CTR1_SDW_LOG2, ilog2(sdw));
> + _ctr1 |= FIELD_PREP(CTR1_SBL_1, sbl_max - 1);
> + _ctr1 |= FIELD_PREP(CTR1_DDW_LOG2, ilog2(ddw));
> + _ctr1 |= FIELD_PREP(CTR1_DBL_1, dbl_max - 1);
> +
> + if (ddw != sdw) {
> + _ctr1 |= FIELD_PREP(CTR1_PAM, CTR1_PAM_PACK_UNPACK);
> + /* Should never reach this case as ddw is clamped down 
> */
> + if (len & (ddw - 1)) {
> + dev_err(chan2dev(chan),
> + "Packing mode is enabled and len is not 
> multiple of ddw\n");
> + return -EINVAL;
> + }
> + }
> +
> + /* dst = mem */
> + _ctr2 &= ~CTR2_DREQ;
> +
> + break;
> +
> + default:
> + dev_err(chan2dev(chan), "Direction %s not supported\n",
> + dmaengine_get_direction_text(dir));
> + return -EINVAL;
> + }
> +
> + *ccr |= FIELD_PREP(CCR_PRIO, FIELD_GET(STM32_DMA3_DT_PRIO, ch_conf));
> + *ctr1 = _ctr1;
> + *ctr2 = _ctr2;
> +
> + dev_dbg(chan2dev(chan), "%s: sdw=%u bytes sbl=%u beats ddw=%u bytes 
> dbl=%u beats\n",
> + __func__, sdw, sbl_max, ddw, dbl_max);
> +
> + return 0;
> +}
> +
> +static void stm32_dma3_chan_start(struct stm32_dma3_chan *chan)
> +{
> + struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan);
> + struct virt_dma_desc *vdesc;
> + struct stm32_dma3_hwdesc *hwdesc;
> + u32 id = chan->id;
> + u32 csr, ccr;
> +
> + vdesc = vchan_next_desc(>vchan);
> + if (!vdesc) {
> + chan->swdesc = NULL;
> + return;
> + }
> + list_del(>node);
> +
> + chan-&g

Re: [PATCH 05/12] dmaengine: Add STM32 DMA3 support

[Frank Li]

On Mon, May 13, 2024 at 11:21:18AM +0200, Amelie Delaunay wrote:
> Hi Frank,
> 
> On 5/7/24 22:26, Frank Li wrote:
> > On Tue, May 07, 2024 at 01:33:31PM +0200, Amelie Delaunay wrote:
> > > Hi Vinod,
> > > 
> > > Thanks for the review.
> > > 
> > > On 5/4/24 14:40, Vinod Koul wrote:
> > > > On 23-04-24, 14:32, Amelie Delaunay wrote:
> > > > > STM32 DMA3 driver supports the 3 hardware configurations of the STM32 
> > > > > DMA3
> > > > > controller:
> > > > > - LPDMA (Low Power): 4 channels, no FIFO
> > > > > - GPDMA (General Purpose): 16 channels, FIFO from 8 to 32 bytes
> > > > > - HPDMA (High Performance): 16 channels, FIFO from 8 to 256 bytes
> > > > > Hardware configuration of the channels is retrieved from the hardware
> > > > > configuration registers.
> > > > > The client can specify its channel requirements through device tree.
> > > > > STM32 DMA3 channels can be individually reserved either because they 
> > > > > are
> > > > > secure, or dedicated to another CPU.
> > > > > Indeed, channels availability depends on Resource Isolation Framework
> > > > > (RIF) configuration. RIF grants access to buses with Compartiment ID
> > > > 
> > > > Compartiment? typo...?
> > > > 
> > > 
> > > Sorry, indeed, Compartment instead.
> > > 
> > > > > (CIF) filtering, secure and privilege level. It also assigns DMA 
> > > > > channels
> > > > > to one or several processors.
> > > > > DMA channels used by Linux should be CID-filtered and statically 
> > > > > assigned
> > > > > to CID1 or shared with other CPUs but using semaphore. In case CID
> > > > > filtering is not configured, dma-channel-mask property can be used to
> > > > > specify available DMA channels to the kernel, otherwise such channels
> > > > > will be marked as reserved and can't be used by Linux.
> > > > > 
> > > > > Signed-off-by: Amelie Delaunay 
> > > > > ---
> > > > >drivers/dma/stm32/Kconfig  |   10 +
> > > > >drivers/dma/stm32/Makefile |1 +
> > > > >drivers/dma/stm32/stm32-dma3.c | 1431 
> > > > > 
> > > > >3 files changed, 1442 insertions(+)
> > > > >create mode 100644 drivers/dma/stm32/stm32-dma3.c
> > > > > 
> > > > > diff --git a/drivers/dma/stm32/Kconfig b/drivers/dma/stm32/Kconfig
> > > > > index b72ae1a4502f..4d8d8063133b 100644
> > > > > --- a/drivers/dma/stm32/Kconfig
> > > > > +++ b/drivers/dma/stm32/Kconfig
> > > > > @@ -34,4 +34,14 @@ config STM32_MDMA
> > > > > If you have a board based on STM32 SoC with such DMA 
> > > > > controller
> > > > > and want to use MDMA say Y here.
> > > > > +config STM32_DMA3
> > > > > + tristate "STMicroelectronics STM32 DMA3 support"
> > > > > + select DMA_ENGINE
> > > > > + select DMA_VIRTUAL_CHANNELS
> > > > > + help
> > > > > +   Enable support for the on-chip DMA3 controller on 
> > > > > STMicroelectronics
> > > > > +   STM32 platforms.
> > > > > +   If you have a board based on STM32 SoC with such DMA3 
> > > > > controller
> > > > > +   and want to use DMA3, say Y here.
> > > > > +
> > > > >endif
> > > > > diff --git a/drivers/dma/stm32/Makefile b/drivers/dma/stm32/Makefile
> > > > > index 663a3896a881..5082db4b4c1c 100644
> > > > > --- a/drivers/dma/stm32/Makefile
> > > > > +++ b/drivers/dma/stm32/Makefile
> > > > > @@ -2,3 +2,4 @@
> > > > >obj-$(CONFIG_STM32_DMA) += stm32-dma.o
> > > > >obj-$(CONFIG_STM32_DMAMUX) += stm32-dmamux.o
> > > > >obj-$(CONFIG_STM32_MDMA) += stm32-mdma.o
> > > > > +obj-$(CONFIG_STM32_DMA3) += stm32-dma3.o
> > > > 
> > > > are there any similarities in mdma/dma and dma3..?
> > > > can anything be reused...?
> > > > 
> > > 
> > > DMA/MDMA were originally intended for STM32 MCUs and have been used in
> > > STM32MP1 MPUs.
> > > New MPUs (STM32MP2, ...) and STM32 MCUs (STM32H5, STM32

Re: gwt-maven-springboot-archetype updated ...

['Frank Hossfeld' via GWT Users]

Bott -> Boot ...  (spelling correction  arrrgh)

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit/8726885d-cac3-43ff-8bc4-cc0353aeab44n%40googlegroups.com.

Re: gwt-maven-springboot-archetype updated ...

['Frank Hossfeld' via GWT Users]

and the version of Spring Bott is updated to 3.2.5

Frank Hossfeld schrieb am Mittwoch, 15. Mai 2024 um 20:36:10 UTC+2:

> New version available ... This one fixes the serializationPolicyFilePath 
> issue ... 
>
> Craig Mitchell schrieb am Montag, 4. März 2024 um 12:59:57 UTC+1:
>
>> Thank you Frank for making the excellent tool!  Just checked the changes, 
>> working perfectly.  
>>
>> On Monday 4 March 2024 at 7:55:58 pm UTC+11 Frank Hossfeld wrote:
>>
>>> Thanks Craig for contributing. New version is online.
>>>
>>> Frank Hossfeld schrieb am Dienstag, 6. Februar 2024 um 08:11:15 UTC+1:
>>>
>>>> new version is online ... 
>>>>
>>>> Craig Mitchell schrieb am Montag, 5. Februar 2024 um 02:13:45 UTC+1:
>>>>
>>>>> Done:  
>>>>> https://github.com/NaluKit/gwt-maven-springboot-archetype/issues/4  
>>>>> Cheers.
>>>>>
>>>>> On Monday 5 February 2024 at 3:59:55 am UTC+11 Frank Hossfeld wrote:
>>>>>
>>>>>> Please can you open an issue regarding the gwt-servlet-jakarta 
>>>>>> problem ...
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Craig Mitchell schrieb am Sonntag, 4. Februar 2024 um 04:57:34 UTC+1:
>>>>>>
>>>>>>> I noticed that it always logs the error:
>>>>>>>
>>>>>>> ERROR: The serialization policy file '/mywebapp/xxx.gwt.rpc' was not 
>>>>>>> found; did you forget to include it in this deployment?
>>>>>>>
>>>>>>> on the first RPC call.  The RPC still seems to work fine, but 
>>>>>>> wondering if this is an issue or not?
>>>>>>>
>>>>>>> Also, should this be "gwt-servlet-jakarta", not "gwt-servlet"?  
>>>>>>> https://github.com/NaluKit/gwt-maven-springboot-archetype/blob/main/modular-springboot-webapp/src/test/resources/projects/basic-webapp/reference/basic-webapp-shared/pom.xml#L16
>>>>>>>  
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> On Saturday 3 February 2024 at 11:10:34 am UTC+11 Craig Mitchell 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Just realised you can also set it in your server pom.xml, in the 
>>>>>>>> Spring Boot Maven plugin (configuration > jvmArguments), instead of 
>>>>>>>> the 
>>>>>>>> launcher:
>>>>>>>>
>>>>>>>> 
>>>>>>>>   org.springframework.boot
>>>>>>>>   spring-boot-maven-plugin
>>>>>>>>   
>>>>>>>> 
>>>>>>>>   
>>>>>>>> repackage
>>>>>>>>   
>>>>>>>> 
>>>>>>>>   
>>>>>>>>   
>>>>>>>> false
>>>>>>>> 
>>>>>>>>   
>>>>>>>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000
>>>>>>>> 
>>>>>>>>   
>>>>>>>> 
>>>>>>>>
>>>>>>>> On Friday 2 February 2024 at 9:54:46 am UTC+11 Craig Mitchell wrote:
>>>>>>>>
>>>>>>>>> Thanks Thomas.  Now switched to:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -Dspring-boot.run.jvmArguments=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000
>>>>>>>>>
>>>>>>>>> [image: Screen.png]
>>>>>>>>>
>>>>>>>>> Which also works great.  
>>>>>>>>>
>>>>>>>>> On Thursday 1 February 2024 at 7:44:19 pm UTC+11 Thomas Broyer 
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> You may want to use -agentlib:jdwp (as given by IntelliJ IDEA) 
>>>>>>>>>> rather than the legacy -Xdebug -Xrunjdwp (and then you no longer 
>>>>>>>>>> need the 
>>>>>>>>>> quotes )
>>>>>>>>>>
>>>>>>>>>> And when I say legacy, I really mean it: already in Java 8 
>>>>>>>>>> -Xdebug does nothing (
>>>>>>>>>> https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#BABHDABI)
>>>>>>>>>>  
>>>>>>>>>> and Xrunjdwp is superceded by agentlib (
>>>>>>>>>> https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#BABDCEGG
>>>>>>>>>> )
>>>>>>>>>>
>>>>>>>>>> On Wednesday, January 31, 2024 at 10:49:41 PM UTC+1 
>>>>>>>>>> ma...@craig-mitchell.com wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank you!  Working great now (quotes were important).  
>>>>>>>>>>>
>>>>>>>>>>> [image: Screen.png]
>>>>>>>>>>>
>>>>>>>>>>> On Thursday 1 February 2024 at 6:38:47 am UTC+11 Frank Hossfeld 
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mmmh, it might be worth adding this information to the 
>>>>>>>>>>>> archetype docs.
>>>>>>>>>>>>
>>>>>>>>>>>>

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit/dbeab6cf-355f-46da-a67b-0035f8ae6c0an%40googlegroups.com.

Re: gwt-maven-springboot-archetype updated ...

['Frank Hossfeld' via GWT Users]

New version available ... This one fixes the serializationPolicyFilePath 
issue ... 

Craig Mitchell schrieb am Montag, 4. März 2024 um 12:59:57 UTC+1:

> Thank you Frank for making the excellent tool!  Just checked the changes, 
> working perfectly.  
>
> On Monday 4 March 2024 at 7:55:58 pm UTC+11 Frank Hossfeld wrote:
>
>> Thanks Craig for contributing. New version is online.
>>
>> Frank Hossfeld schrieb am Dienstag, 6. Februar 2024 um 08:11:15 UTC+1:
>>
>>> new version is online ... 
>>>
>>> Craig Mitchell schrieb am Montag, 5. Februar 2024 um 02:13:45 UTC+1:
>>>
>>>> Done:  
>>>> https://github.com/NaluKit/gwt-maven-springboot-archetype/issues/4  
>>>> Cheers.
>>>>
>>>> On Monday 5 February 2024 at 3:59:55 am UTC+11 Frank Hossfeld wrote:
>>>>
>>>>> Please can you open an issue regarding the gwt-servlet-jakarta problem 
>>>>> ...
>>>>>
>>>>> Thanks
>>>>>
>>>>> Craig Mitchell schrieb am Sonntag, 4. Februar 2024 um 04:57:34 UTC+1:
>>>>>
>>>>>> I noticed that it always logs the error:
>>>>>>
>>>>>> ERROR: The serialization policy file '/mywebapp/xxx.gwt.rpc' was not 
>>>>>> found; did you forget to include it in this deployment?
>>>>>>
>>>>>> on the first RPC call.  The RPC still seems to work fine, but 
>>>>>> wondering if this is an issue or not?
>>>>>>
>>>>>> Also, should this be "gwt-servlet-jakarta", not "gwt-servlet"?  
>>>>>> https://github.com/NaluKit/gwt-maven-springboot-archetype/blob/main/modular-springboot-webapp/src/test/resources/projects/basic-webapp/reference/basic-webapp-shared/pom.xml#L16
>>>>>>  
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> On Saturday 3 February 2024 at 11:10:34 am UTC+11 Craig Mitchell 
>>>>>> wrote:
>>>>>>
>>>>>>> Just realised you can also set it in your server pom.xml, in the 
>>>>>>> Spring Boot Maven plugin (configuration > jvmArguments), instead of the 
>>>>>>> launcher:
>>>>>>>
>>>>>>> 
>>>>>>>   org.springframework.boot
>>>>>>>   spring-boot-maven-plugin
>>>>>>>   
>>>>>>> 
>>>>>>>   
>>>>>>> repackage
>>>>>>>   
>>>>>>> 
>>>>>>>   
>>>>>>>   
>>>>>>> false
>>>>>>> 
>>>>>>>   
>>>>>>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000
>>>>>>> 
>>>>>>>   
>>>>>>> 
>>>>>>>
>>>>>>> On Friday 2 February 2024 at 9:54:46 am UTC+11 Craig Mitchell wrote:
>>>>>>>
>>>>>>>> Thanks Thomas.  Now switched to:
>>>>>>>>
>>>>>>>>
>>>>>>>> -Dspring-boot.run.jvmArguments=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000
>>>>>>>>
>>>>>>>> [image: Screen.png]
>>>>>>>>
>>>>>>>> Which also works great.  
>>>>>>>>
>>>>>>>> On Thursday 1 February 2024 at 7:44:19 pm UTC+11 Thomas Broyer 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> You may want to use -agentlib:jdwp (as given by IntelliJ IDEA) 
>>>>>>>>> rather than the legacy -Xdebug -Xrunjdwp (and then you no longer need 
>>>>>>>>> the 
>>>>>>>>> quotes )
>>>>>>>>>
>>>>>>>>> And when I say legacy, I really mean it: already in Java 8 -Xdebug 
>>>>>>>>> does nothing (
>>>>>>>>> https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#BABHDABI)
>>>>>>>>>  
>>>>>>>>> and Xrunjdwp is superceded by agentlib (
>>>>>>>>> https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#BABDCEGG
>>>>>>>>> )
>>>>>>>>>
>>>>>>>>> On Wednesday, January 31, 2024 at 10:49:41 PM UTC+1 
>>>>>>>>> ma...@craig-mitchell.com wrote:
>>>>>>>>>
>>>>>>>>>> Thank you!  Working great now (quotes were important).  
>>>>>>>>>>
>>>>>>>>>> [image: Screen.png]
>>>>>>>>>>
>>>>>>>>>> On Thursday 1 February 2024 at 6:38:47 am UTC+11 Frank Hossfeld 
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Mmmh, it might be worth adding this information to the archetype 
>>>>>>>>>>> docs.
>>>>>>>>>>>
>>>>>>>>>>>

-- 
You received this message because you are subscribed to the Google Groups "GWT 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-web-toolkit/d6b24035-3b55-475c-b6ea-aafa47f69fd2n%40googlegroups.com.

Re: Linux 6.9

[Frank Scheiner]

Dear all,

here comes the usual update on Linux/ia64:

The reason for the userland regression we mentioned last time (in [1])
was found and fixed shortly after the release of v6.8.

[1]:
https://lore.kernel.org/all/145da253-b3bc-43da-a262-a3ebdfbea...@web.de/

Furthermore there were no new hard regressions detected in addition to
what was reported in [2] already. If you have an ia64 machine with more
than 64 hardware threads and want to run Linux on it, get in touch with
us. :-)

[2]:
https://lore.kernel.org/linux-ia64/cahtyxddy5lub_uemqrgr8o_g-xk0_xrd3j7wvb9t9rrd5x6...@mail.gmail.com/

Again all ia64 machines (see [3] for a list) and platforms (HP Sim on
Ski) we have available for testing continue to work, no system support
was lost during this cycle.

[3]:
https://lore.kernel.org/all/fe5f6e9b-02a2-42e9-8151-ae4b6fdba...@web.de/

In the meantime gcc-14 was released, meaning that the regular
compilation and testing of Linux mainline release (candidates) switched
to gcc-15 snapshots now (starting with v6.9-rc6). Enabling LRA for the
cross-compiler continues to make **no problems** for ia64 kernels. The
same is true with the switch to binutils 2.42 since v6.9-rc2.



Last time ([1]) we had to report about an approaching decrease in the
number of available Linux distributions with support for ia64. This was
sad to report, also because options are important.

But don't worry, the distro options for your ia64 gear just have
increased again:

Enter **EPIC Slack** ([4]) - an unofficial "port" of Slackware for ia64
that was started recently and - though still work in progress - is
already network booting on all test machines available to us. If you're
too young to know what Slackware is, head over to [5] and learn more
about it (-;.

[4]: http://epic-slack.org/

[5]: http://www.slackware.com/



Thank you all for your hard work on Linux!

Cheers,
Frank et al

RE: [chrony-users] Understand why system clock is bad even though chrony offsets look fine

[MUZZULINI Frank]

If you have no control of the server, how do you even know that your time is 
off and not the server’s time? An application should never crash due to an 
unexpected timestamp in a received packet, that’s bad programming.

Apart from that, if your machine synchronises itself over the internet, 50us is 
well within the accuracy that you can achieve. If you really need to compare 
timestamps with microsec resolution, both systems should synchronise to the 
same local NTP source, preferably a GPS receiver.

Regards,
Frank

From: Abhijith Sethuraj 
Sent: Tuesday, 14 May, 2024 1:05 PM
To: chrony-users@chrony.tuxfamily.org
Subject: Re: [chrony-users] Understand why system clock is bad even though 
chrony offsets look fine

*EXTERNAL source*

> How did you measure that 50us error? Can you use that source of time
> for synchronization?
Unfortunately, we can't use that as a source. This difference is what an 
application noticed when it was looking at a timestamp that's there inside a 
data packet (think of it as tx timestamp from a server that we don't have 
control over) and what's obtained from system clock via gettimeofday() right 
when it received the data packet. The reason why we noticed this was because 
the timestamp from system clock turned out to be lesser than the tx timestamp, 
which makes the app crash. The network delays through switches and servers 
likely come to around 50us, which is how we measured it. What do you think is a 
good indication to catch these issues? Should we look at frequency/freq 
adjustment of system clock? So far we were only relying on offsets from ref 
clock, but this issue made us think of time error (root distance) as well. Is 
there anything else that you would recommend we monitor on our end to catch 
these issues? Also, if root distance is ~1ms, does that imply that the value of 
current time that I get from gettimeofday() essentially has an error of ~1ms or 
is this just inferring to the error in measurements from source (and that the 
user-space app need not consider this)?


Thanks,
Abhijith



On Mon, Apr 22, 2024 at 12:05 PM Miroslav Lichvar 
mailto:mlich...@redhat.com>> wrote:
On Fri, Apr 19, 2024 at 02:44:21AM +0530, Abhijith Sethuraj wrote:
> Hello,
>
> I'm noticing issues with my system clock being inaccurate by almost 50us,
> even though "System time" in `chronyc tracking` shows offsets in the order
> of ns. This was noticed by an application that tried to get current time by
> calling `gettimeofday()`.

How did you measure that 50us error? Can you use that source of time
for synchronization?

> Root delay  : 0.73557 seconds
>
> Root dispersion : 0.000997235 seconds

Root distance (half of root delay + root dispersion) is over 1
millisecond here. That's an estimate of maximum clock error due to
asymmetries and clock instability.

> almost similar to "root dispersion"? Also, what recommendations do you have
> for monitoring chrony, so that I can catch this before it affects my app?
> Also, are there any config tweaks that I can try out here to help me?

You need a better server, one that has a smaller root dispersion and
hardware timestamping on both the server and client to get the maximum
error below 50 microseconds.

--
Miroslav Lichvar


--
To unsubscribe email 
chrony-users-requ...@chrony.tuxfamily.org<mailto:chrony-users-requ...@chrony.tuxfamily.org>
with "unsubscribe" in the subject.
For help email 
chrony-users-requ...@chrony.tuxfamily.org<mailto:chrony-users-requ...@chrony.tuxfamily.org>
with "help" in the subject.
Trouble?  Email 
listmas...@chrony.tuxfamily.org<mailto:listmas...@chrony.tuxfamily.org>.

Re: [FRIAM] The Rise of the Maya Civilization

[Frank Wimberly]

In tenth grade I wrote a term paper with the title The Language, Arithmetic
and Astronomy of the Ancient Maya.  I doubt that it was a deep analysis
given that it was ten typewritten pages.

---
Frank C. Wimberly
140 Calle Ojo Feliz,
Santa Fe, NM 87505

505 670-9918
Santa Fe, NM

On Tue, May 14, 2024, 2:45 PM Jochen Fromm  wrote:

> Takeshi Inomata from the University of Arizona does interesting work on
> the rise of the Maya civilization:
>
> Monumental architecture at Aguada Fenix and the rise of Maya civilization,
> Nature 582 (2020) 530-533
>
> https://pasolibre.grecu.mx/wp-content/uploads/2020/07/41586_2020_2343_opt.pdf
>
> -J.
>
> -. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. .
> FRIAM Applied Complexity Group listserv
> Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoom
> https://bit.ly/virtualfriam
> to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
> FRIAM-COMIC http://friam-comic.blogspot.com/
> archives:  5/2017 thru present
> https://redfish.com/pipermail/friam_redfish.com/
>   1/2003 thru 6/2021  http://friam.383.s1.nabble.com/
>
-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. .
FRIAM Applied Complexity Group listserv
Fridays 9a-12p Friday St. Johns Cafe   /   Thursdays 9a-12p Zoom 
https://bit.ly/virtualfriam
to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
FRIAM-COMIC http://friam-comic.blogspot.com/
archives:  5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/
  1/2003 thru 6/2021  http://friam.383.s1.nabble.com/

Re: [gentoo-user] Encrypted drives, password generation and management howto, guide.

[Frank Steinmetzger]

Am Tue, May 14, 2024 at 06:28:17AM -0500 schrieb Dale:
> Howdy,
> […]
> remember either, or write notes to remember them.  I also wanted to
> avoid the desktop copy and paste, or clipboard, mechanism.  I'm not sure
> how that data is stored in the clipboard and how good it is at erasing
> it when I clear it.

The mark-and-middleclick you describe further down is the very same as the 
“normal” clipboard. It is just accessed differently.

> First, I needed to generate a password.  I googled, a lot.  I had
> trouble finding a way to generate the type of passwords I wanted but I
> finally found one.

Care to elaborate regarding the “password you wanted”? There is the obvious 
pwgen, which can generate passwords with given character sets and length. 
Keepass can do this, too, so I assume, Bitwarden (which you use) has a 
similar function.

And if you don’t like parts of the generated PW, keep the part you like, 
generate new and pick the part you like again. Or just let pwgen generate a 
big bunch and pick what you like best from the output.

> […]
> Now that I have a password, how do I keep track of them?  I did some
> more searching.  I wanted something that was command line not GUI. 
> After all, I have BitWarden for websites and such already.  Thing is,
> it's GUI since it is a Firefox add-on.  I'd need to use the clipboard to
> copy and paste.  I want to avoid that remember?  I also wanted something
> that is on its own, separate from my main password tool BitWarden.  I
> found kpcli in the tree.

I didn’t know about kpcli and it is not available in Arch. So I looked it 
up. Turns out it is a non-graphical Keepass client (that’s what the kp 
stands for, after all).

Interestingly, there is also a bitwarden CLI client.

Did you know Keepass (the graphical one) has an autotype feature? This means 
that it simulates the pressing of keys, so it bypasses the clipboard 
entirely. Another advantage of that is that you can set up custom key 
sequences in the autotype field, so you can for example say “first enter the 
username, then press enter, then wait for a second, then enter the password 
and press enter again.” Useful for sites that use a dynamic login screen 
with animations or non-standard input fields.

> Then I needed some way to handle if the password file kpcli uses got
> lost or damaged.  If I were to lose that file, all drives and the data
> on them is lost.  I'd lose everything because there is no way to
> remember the password.

The obvious answer is: backup – encrypted or not. ;-)
My Keepass database is a simple file in my home that is backed up together 
with all the other home files by Borg. Meaning I even have a versioned 
backup of my passwords. Needless to say my backup drives are LUKSed with a 
long passphrase that I have never ever once written down anywhere on paper. 
I’ve been using it for so long now and on several drives, that it is 
ingrained in my brain.

> The kpcli file itself appears to be encrypted. 
> So, it protects itself.  That's good.  I don't need to put the file on
> something that is also encrypted, just copy it to a plain file system as
> it is.  I have a USB stick that I store things on.  Things like drive
> info, what drives go to what volume group, what drive has the OS on it
> etc and the portage world file on it.  I also have some scripts in /root
> that I don't want to lose either so I copy them to the stick as well. 

Be mindful that USB sticks aren’t very reliable. The flash chips in them are 
what is left after quality control deemed them unfit for duty in SSDs (first 
tier) and memory cards (second tier). So always keep several copies, 
possibly on different types of storage media (HDDs, SSDs, optical, whatever).

> Then one important file, my file that contains frequently used
> commands.  It is rather lengthy and is 15 years or more of additions.  I
> copied all that info to a USB stick.  It lives in the fire safe.

TBH, I wouldn’t put all my horses on one USB stick in a fire safe. (Or 
however the saying goes) After a flimsy USB stick with questionable flash 
chips has been subjected to high temperatures for a longer time, chances are 
you may not be able to access its data ever again.

> How I use all this.  I do this in a Konsole, within KDE, which has
> tabs.  Might work on a plain console to tho.  If I need to open a
> encrypted drive, or set of drives, I open kpcli and get it to show the
> password for that drive in one tab.  I then run the little script to
> open and mount that drive in another tab.  When it asks for the
> password, I highlight the password from kpcli tab and then switch tabs
> and middle click to paste the password in.

Since you’ve already scripted most of it, you could possible go the full 
way. Use the HDD’s UUID as key and either store the password in a file that 
is named with the UUID, or in keepass with the UUID as entry title. Then you 
can let the script retrieve the password all by itself without any need for 
copy-pasting – 

[Bug 2045250] Re: pam_lastlog doesn't handle localtime_r related errors properly

[Frank Heimes]

Hello Boris,
unfortunately no, not really any further actions - as of now.
The problem is that bringing it into M, J and F would require a testplan, and 
with that a stable reproducer that we do not have (and couldn't find - even at 
looking at this bug in other distros).
So it was first of all decided to pick it up for noble (since at that time this 
problem was looked at, noble was still in development).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2045250

Title:
  pam_lastlog doesn't handle localtime_r related errors properly

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2045250/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[pve-devel] [PATCH manager v10 08/11] ui: add edit window for dir mappings

[Markus Frank]

Signed-off-by: Markus Frank 
---
 www/manager6/Makefile |   1 +
 www/manager6/window/DirMapEdit.js | 222 ++
 2 files changed, 223 insertions(+)
 create mode 100644 www/manager6/window/DirMapEdit.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 2c3a822b..f6140562 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -137,6 +137,7 @@ JSSRC=  
\
window/TreeSettingsEdit.js  \
window/PCIMapEdit.js\
window/USBMapEdit.js\
+   window/DirMapEdit.js\
window/GuestImport.js   \
ha/Fencing.js   \
ha/GroupEdit.js \
diff --git a/www/manager6/window/DirMapEdit.js 
b/www/manager6/window/DirMapEdit.js
new file mode 100644
index ..cda5824b
--- /dev/null
+++ b/www/manager6/window/DirMapEdit.js
@@ -0,0 +1,222 @@
+Ext.define('PVE.window.DirMapEditWindow', {
+extend: 'Proxmox.window.Edit',
+
+mixins: ['Proxmox.Mixin.CBind'],
+
+cbindData: function(initialConfig) {
+   let me = this;
+   me.isCreate = !me.name;
+   me.method = me.isCreate ? 'POST' : 'PUT';
+   me.hideMapping = !!me.entryOnly;
+   me.hideComment = me.name && !me.entryOnly;
+   me.hideNodeSelector = me.nodename || me.entryOnly;
+   me.hideNode = !me.nodename || !me.hideNodeSelector;
+   return {
+   name: me.name,
+   nodename: me.nodename,
+   };
+},
+
+submitUrl: function(_url, data) {
+   let me = this;
+   let name = me.isCreate ? '' : me.name;
+   return `/cluster/mapping/dir/${name}`;
+},
+
+title: gettext('Add Dir mapping'),
+
+onlineHelp: 'resource_mapping',
+
+method: 'POST',
+
+controller: {
+   xclass: 'Ext.app.ViewController',
+
+   onGetValues: function(values) {
+   let me = this;
+   let view = me.getView();
+   values.node ??= view.nodename;
+
+   let name = values.name;
+   let description = values.description;
+   let xattr = values.xattr;
+   let acl = values.acl;
+   let deletes = values.delete;
+
+   delete values.description;
+   delete values.name;
+   delete values.xattr;
+   delete values.acl;
+
+   let map = [];
+   if (me.originalMap) {
+   map = PVE.Parser.filterPropertyStringList(me.originalMap, (e) 
=> e.node !== values.node);
+   }
+   if (values.path) {
+   map.push(PVE.Parser.printPropertyString(values));
+   }
+
+   values = { map };
+   if (description) {
+   values.description = description;
+   }
+   if (xattr) {
+   values.xattr = xattr;
+   }
+   if (acl) {
+   values.acl = acl;
+   }
+   if (deletes) {
+   values.delete = deletes;
+   }
+
+   if (view.isCreate) {
+   values.id = name;
+   }
+   return values;
+   },
+
+   onSetValues: function(values) {
+   let me = this;
+   let view = me.getView();
+   me.originalMap = [...values.map];
+   let configuredNodes = [];
+   PVE.Parser.filterPropertyStringList(values.map, (e) => {
+   configuredNodes.push(e.node);
+   if (e.node === view.nodename) {
+   values = e;
+   }
+   return false;
+   });
+
+   me.lookup('nodeselector').disallowedNodes = configuredNodes;
+
+   return values;
+   },
+
+   init: function(view) {
+   let me = this;
+
+   if (!view.nodename) {
+   //throw "no nodename given";
+   }
+   },
+},
+
+items: [
+   {
+   xtype: 'inputpanel',
+   onGetValues: function(values) {
+   return this.up('window').getController().onGetValues(values);
+   },
+
+   onSetValues: function(values) {
+   return this.up('window').getController().onSetValues(values);
+   },
+
+   columnT: [
+   {
+   xtype: 'displayfield',
+   reference: 'directory-hint',
+   columnWidth: 1,
+   value: 'Make sure the directory exists.',
+   cbind: {
+   disabled: '{hideMapping}',
+   hidden: '{hideMapping}',
+   },
+   userCls: 'pmx-hint',
+   },
+   ],
+
+   column1: [
+   {
+   xtype: 'pmxDisplayEditField',
+   fieldLabel: gettext('Name'),
+   cbind: {
+

[pve-devel] [PATCH docs v10 3/11] add doc section for the shared filesystem virtio-fs

[Markus Frank]

Signed-off-by: Markus Frank 
---
 qm.adoc | 94 +++--
 1 file changed, 92 insertions(+), 2 deletions(-)

diff --git a/qm.adoc b/qm.adoc
index 42c26db..755e20e 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1081,6 +1081,95 @@ recommended to always use a limiter to avoid guests 
using too many host
 resources. If desired, a value of '0' for `max_bytes` can be used to disable
 all limits.
 
+[[qm_virtiofs]]
+Virtio-fs
+~
+
+Virtio-fs is a shared file system that enables sharing a directory between host
+and guest VM. It takes advantage of the locality of virtual machines and the
+hypervisor to get a higher throughput than the 9p remote file system protocol.
+
+To use virtio-fs, the https://gitlab.com/virtio-fs/virtiofsd[virtiofsd] daemon
+needs to run in the background. In {pve}, this process starts immediately 
before
+the start of QEMU.
+
+Linux VMs with kernel >=5.4 support this feature by default.
+
+There is a guide available on how to utilize virtio-fs in Windows VMs.
+https://github.com/virtio-win/kvm-guest-drivers-windows/wiki/Virtiofs:-Shared-file-system
+
+Known Limitations
+^
+
+* Virtiofsd crashing means no recovery until VM is fully stopped and restarted.
+* Virtiofsd not responding may result in NFS-like hanging access in the VM.
+* Memory hotplug does not work in combination with virtio-fs (also results in
+hanging access).
+* Live migration does not work.
+* Windows cannot understand ACLs. Therefore, disable it for Windows VMs,
+otherwise the virtio-fs device will not be visible within the VMs.
+
+Add Mapping for Shared Directories
+^^
+
+To add a mapping for a shared directory, either use the API directly with
+`pvesh` as described in the xref:resource_mapping[Resource Mapping] section:
+
+
+pvesh create /cluster/mapping/dir --id dir1 \
+--map node=node1,path=/path/to/share1 \
+--map node=node2,path=/path/to/share2,submounts=1 \
+--xattr 1 \
+--acl 1
+
+
+The `acl` parameter automatically implies `xattr`, that is, it makes no
+difference whether you set `xattr` to `0` if `acl` is set to `1`.
+
+Set `submounts` to `1` when multiple file systems are mounted in a shared
+directory to prevent the guest from creating duplicates because of file system
+specific inode IDs that get passed through.
+
+
+Add virtio-fs to a VM
+^
+
+To share a directory using virtio-fs, add the parameter `virtiofs` (N can be
+anything between 0 and 9) to the VM config and use a directory ID (dirid) that
+has been configured in the resource mapping. Additionally, you can set the
+`cache` option to either `always`, `never`, or `auto` (default: `auto`),
+depending on your requirements. How the different caching modes behave can be
+read at https://lwn.net/Articles/774495/ under the title "Caching Modes". To
+enable writeback cache set `writeback` to `1`.
+
+If you want virtio-fs to honor the `O_DIRECT` flag, you can set the `direct-io`
+parameter to `1` (default: `0`). This will degrade performance, but is useful 
if
+applications do their own caching.
+
+Additionally, it is possible to overwrite the default mapping settings for
+`xattr` and `acl` by setting them to either `1` or `0`. The `acl` parameter
+automatically implies `xattr`, that is, it makes no difference whether you set
+`xattr` to `0` if `acl` is set to `1`.
+
+
+qm set  -virtiofs0 dirid=,cache=always,direct-io=1
+qm set  -virtiofs1 ,cache=never,xattr=1
+qm set  -virtiofs2 ,acl=1,writeback=1
+
+
+To mount virtio-fs in a guest VM with the Linux kernel virtio-fs driver, run 
the
+following command inside the guest:
+
+
+mount -t virtiofs  
+
+
+The dirid associated with the path on the current node is also used as the 
mount
+tag (name used to mount the device on the guest).
+
+For more information on available virtiofsd parameters, see the
+https://gitlab.com/virtio-fs/virtiofsd[GitLab virtiofsd project page].
+
 [[qm_bootorder]]
 Device Boot Order
 ~
@@ -1743,8 +1832,9 @@ in the relevant tab in the `Resource Mappings` category, 
or on the cli with
 
 [thumbnail="screenshot/gui-datacenter-mapping-pci-edit.png"]
 
-Where `` is the hardware type (currently either `pci` or `usb`) and
-`` are the device mappings and other configuration parameters.
+Where `` is the hardware type (currently either `pci`, `usb` or
+xref:qm_virtiofs[dir]) and `` are the device mappings and other
+configuration parameters.
 
 Note that the options must include a map property with all identifying
 properties of that hardware, so that it's possible to verify the hardware did
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v10 09/11] ui: ResourceMapTree for DIR

[Markus Frank]

Signed-off-by: Markus Frank 
---
 www/manager6/Makefile |  1 +
 www/manager6/dc/Config.js | 10 +++
 www/manager6/dc/DirMapView.js | 50 +++
 3 files changed, 61 insertions(+)
 create mode 100644 www/manager6/dc/DirMapView.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index f6140562..5a3541e0 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -189,6 +189,7 @@ JSSRC=  
\
dc/RealmSyncJob.js  \
dc/PCIMapView.js\
dc/USBMapView.js\
+   dc/DirMapView.js\
lxc/CmdMenu.js  \
lxc/Config.js   \
lxc/CreateWizard.js \
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index ddbb58b1..3355c835 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -320,6 +320,16 @@ Ext.define('PVE.dc.Config', {
title: gettext('USB Devices'),
flex: 1,
},
+   {
+   xtype: 'splitter',
+   collapsible: false,
+   performCollapse: false,
+   },
+   {
+   xtype: 'pveDcDirMapView',
+   title: gettext('Directories'),
+   flex: 1,
+   },
],
},
);
diff --git a/www/manager6/dc/DirMapView.js b/www/manager6/dc/DirMapView.js
new file mode 100644
index ..4468e951
--- /dev/null
+++ b/www/manager6/dc/DirMapView.js
@@ -0,0 +1,50 @@
+Ext.define('pve-resource-dir-tree', {
+extend: 'Ext.data.Model',
+idProperty: 'internalId',
+fields: ['type', 'text', 'path', 'id', 'description', 'digest'],
+});
+
+Ext.define('PVE.dc.DirMapView', {
+extend: 'PVE.tree.ResourceMapTree',
+alias: 'widget.pveDcDirMapView',
+
+editWindowClass: 'PVE.window.DirMapEditWindow',
+baseUrl: '/cluster/mapping/dir',
+mapIconCls: 'fa fa-folder',
+entryIdProperty: 'path',
+
+store: {
+   sorters: 'text',
+   model: 'pve-resource-dir-tree',
+   data: {},
+},
+
+columns: [
+   {
+   xtype: 'treecolumn',
+   text: gettext('ID/Node'),
+   dataIndex: 'text',
+   width: 200,
+   },
+   {
+   text: gettext('xattr'),
+   dataIndex: 'xattr',
+   },
+   {
+   text: gettext('acl'),
+   dataIndex: 'acl',
+   },
+   {
+   text: gettext('submounts'),
+   dataIndex: 'submounts',
+   },
+   {
+   header: gettext('Comment'),
+   dataIndex: 'description',
+   renderer: function(value, _meta, record) {
+   return value ?? record.data.comment;
+   },
+   flex: 1,
+   },
+],
+});
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v10 10/11] ui: form: add DIRMapSelector

[Markus Frank]

Signed-off-by: Markus Frank 
---
 www/manager6/Makefile   |  1 +
 www/manager6/form/DirMapSelector.js | 63 +
 2 files changed, 64 insertions(+)
 create mode 100644 www/manager6/form/DirMapSelector.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 5a3541e0..cac8cd02 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -35,6 +35,7 @@ JSSRC=
\
form/ContentTypeSelector.js \
form/ControllerSelector.js  \
form/DayOfWeekSelector.js   \
+   form/DirMapSelector.js  \
form/DiskFormatSelector.js  \
form/DiskStorageSelector.js \
form/FileSelector.js\
diff --git a/www/manager6/form/DirMapSelector.js 
b/www/manager6/form/DirMapSelector.js
new file mode 100644
index ..473a2ffe
--- /dev/null
+++ b/www/manager6/form/DirMapSelector.js
@@ -0,0 +1,63 @@
+Ext.define('PVE.form.DirMapSelector', {
+extend: 'Proxmox.form.ComboGrid',
+alias: 'widget.pveDirMapSelector',
+
+store: {
+   fields: ['name', 'path'],
+   filterOnLoad: true,
+   sorters: [
+   {
+   property: 'id',
+   direction: 'ASC',
+   },
+   ],
+},
+
+allowBlank: false,
+autoSelect: false,
+displayField: 'id',
+valueField: 'id',
+
+listConfig: {
+   columns: [
+   {
+   header: gettext('Directory ID'),
+   dataIndex: 'id',
+   flex: 1,
+   },
+   {
+   header: gettext('Comment'),
+   dataIndex: 'description',
+   flex: 1,
+   },
+   ],
+},
+
+setNodename: function(nodename) {
+   var me = this;
+
+   if (!nodename || me.nodename === nodename) {
+   return;
+   }
+
+   me.nodename = nodename;
+
+   me.store.setProxy({
+   type: 'proxmox',
+   url: `/api2/json/cluster/mapping/dir?check-node=${nodename}`,
+   });
+
+   me.store.load();
+},
+
+initComponent: function() {
+   var me = this;
+
+   var nodename = me.nodename;
+   me.nodename = undefined;
+
+me.callParent();
+
+   me.setNodename(nodename);
+},
+});
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH cluster/guest-common/docs/qemu-server/manager v10 0/11] virtiofs

[Markus Frank]

Virtio-fs is a shared file system that enables sharing a directory
between host and guest VMs. It takes advantage of the locality of
virtual machines and the hypervisor to get a higher throughput than
the 9p remote file system protocol.

build-order:
1. cluster
2. guest-common
3. docs
4. qemu-server
5. manager

I did not get virtiofsd to run with run_command without creating
zombie processes after stutdown. So I replaced run_command with exec
for now. Maybe someone can find out why this happens.


changes v10:
* rebase to master
* added gui patches again

cluster:

Markus Frank (1):
  add mapping/dir.cfg for resource mapping

 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)


guest-common:

Markus Frank (1):
  add dir mapping section config

 src/Makefile   |   1 +
 src/PVE/Mapping/Dir.pm | 205 +
 2 files changed, 206 insertions(+)
 create mode 100644 src/PVE/Mapping/Dir.pm


docs:

Markus Frank (1):
  add doc section for the shared filesystem virtio-fs

 qm.adoc | 94 +++--
 1 file changed, 92 insertions(+), 2 deletions(-)


qemu-server:

Markus Frank (3):
  add virtiofsd as runtime dependency for qemu-server
  fix #1027: virtio-fs support
  migration: check_local_resources for virtiofs

 PVE/API2/Qemu.pm |  39 ++-
 PVE/QemuServer.pm|  29 -
 PVE/QemuServer/Makefile  |   3 +-
 PVE/QemuServer/Memory.pm |  34 --
 PVE/QemuServer/Virtiofs.pm   | 212 +++
 debian/control   |   1 +
 test/MigrationTest/Shared.pm |   7 ++
 7 files changed, 313 insertions(+), 12 deletions(-)
 create mode 100644 PVE/QemuServer/Virtiofs.pm


manager:

Markus Frank (5):
  api: add resource map api endpoints for directories
  ui: add edit window for dir mappings
  ui: ResourceMapTree for DIR
  ui: form: add DIRMapSelector
  ui: add options to add virtio-fs to qemu config

 PVE/API2/Cluster/Mapping.pm |   7 +
 PVE/API2/Cluster/Mapping/Dir.pm | 317 
 PVE/API2/Cluster/Mapping/Makefile   |   1 +
 www/manager6/Makefile   |   4 +
 www/manager6/Utils.js   |   1 +
 www/manager6/dc/Config.js   |  10 +
 www/manager6/dc/DirMapView.js   |  50 +
 www/manager6/form/DirMapSelector.js |  63 ++
 www/manager6/qemu/HardwareView.js   |  19 ++
 www/manager6/qemu/VirtiofsEdit.js   | 137 
 www/manager6/window/DirMapEdit.js   | 222 +++
 11 files changed, 831 insertions(+)
 create mode 100644 PVE/API2/Cluster/Mapping/Dir.pm
 create mode 100644 www/manager6/dc/DirMapView.js
 create mode 100644 www/manager6/form/DirMapSelector.js
 create mode 100644 www/manager6/qemu/VirtiofsEdit.js
 create mode 100644 www/manager6/window/DirMapEdit.js

-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v10 6/11] migration: check_local_resources for virtiofs

[Markus Frank]

add dir mapping checks to check_local_resources

Since the VM needs to be powered off for migration, migration should
work with a directory on shared storage with all caching settings.

Signed-off-by: Markus Frank 
---
 PVE/QemuServer.pm| 10 +-
 test/MigrationTest/Shared.pm |  7 +++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index a0600d7..ee7699f 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2573,6 +2573,7 @@ sub check_local_resources {
 my $nodelist = PVE::Cluster::get_nodelist();
 my $pci_map = PVE::Mapping::PCI::config();
 my $usb_map = PVE::Mapping::USB::config();
+my $dir_map = PVE::Mapping::Dir::config();
 
 my $missing_mappings_by_node = { map { $_ => [] } @$nodelist };
 
@@ -2584,6 +2585,8 @@ sub check_local_resources {
$entry = PVE::Mapping::PCI::get_node_mapping($pci_map, $id, 
$node);
} elsif ($type eq 'usb') {
$entry = PVE::Mapping::USB::get_node_mapping($usb_map, $id, 
$node);
+   } elsif ($type eq 'dir') {
+   $entry = PVE::Mapping::Dir::get_node_mapping($dir_map, $id, 
$node);
}
if (!scalar($entry->@*)) {
push @{$missing_mappings_by_node->{$node}}, $key;
@@ -2612,9 +2615,14 @@ sub check_local_resources {
push @$mapped_res, $k;
}
}
+   if ($k =~ m/^virtiofs/) {
+   my $entry = parse_property_string('pve-qm-virtiofs', $conf->{$k});
+   $add_missing_mapping->('dir', $k, $entry->{dirid});
+   push @$mapped_res, $k;
+   }
# sockets are safe: they will recreated be on the target side 
post-migrate
next if $k =~ m/^serial/ && ($conf->{$k} eq 'socket');
-   push @loc_res, $k if $k =~ m/^(usb|hostpci|serial|parallel)\d+$/;
+   push @loc_res, $k if $k =~ 
m/^(usb|hostpci|serial|parallel|virtiofs)\d+$/;
 }
 
 die "VM uses local resources\n" if scalar @loc_res && !$noerr;
diff --git a/test/MigrationTest/Shared.pm b/test/MigrationTest/Shared.pm
index aa7203d..c5d0722 100644
--- a/test/MigrationTest/Shared.pm
+++ b/test/MigrationTest/Shared.pm
@@ -90,6 +90,13 @@ $mapping_pci_module->mock(
 },
 );
 
+our $mapping_dir_module = Test::MockModule->new("PVE::Mapping::Dir");
+$mapping_dir_module->mock(
+config => sub {
+   return {};
+},
+);
+
 our $ha_config_module = Test::MockModule->new("PVE::HA::Config");
 $ha_config_module->mock(
 vm_is_ha_managed => sub {
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v10 07/11] api: add resource map api endpoints for directories

[Markus Frank]

Signed-off-by: Markus Frank 
---
 PVE/API2/Cluster/Mapping.pm   |   7 +
 PVE/API2/Cluster/Mapping/Dir.pm   | 317 ++
 PVE/API2/Cluster/Mapping/Makefile |   1 +
 3 files changed, 325 insertions(+)
 create mode 100644 PVE/API2/Cluster/Mapping/Dir.pm

diff --git a/PVE/API2/Cluster/Mapping.pm b/PVE/API2/Cluster/Mapping.pm
index 40386579..9f0dcd2b 100644
--- a/PVE/API2/Cluster/Mapping.pm
+++ b/PVE/API2/Cluster/Mapping.pm
@@ -3,11 +3,17 @@ package PVE::API2::Cluster::Mapping;
 use strict;
 use warnings;
 
+use PVE::API2::Cluster::Mapping::Dir;
 use PVE::API2::Cluster::Mapping::PCI;
 use PVE::API2::Cluster::Mapping::USB;
 
 use base qw(PVE::RESTHandler);
 
+__PACKAGE__->register_method ({
+subclass => "PVE::API2::Cluster::Mapping::Dir",
+path => 'dir',
+});
+
 __PACKAGE__->register_method ({
 subclass => "PVE::API2::Cluster::Mapping::PCI",
 path => 'pci',
@@ -41,6 +47,7 @@ __PACKAGE__->register_method ({
my ($param) = @_;
 
my $result = [
+   { name => 'dir' },
{ name => 'pci' },
{ name => 'usb' },
];
diff --git a/PVE/API2/Cluster/Mapping/Dir.pm b/PVE/API2/Cluster/Mapping/Dir.pm
new file mode 100644
index ..ddb6977d
--- /dev/null
+++ b/PVE/API2/Cluster/Mapping/Dir.pm
@@ -0,0 +1,317 @@
+package PVE::API2::Cluster::Mapping::Dir;
+
+use strict;
+use warnings;
+
+use Storable qw(dclone);
+
+use PVE::INotify;
+use PVE::JSONSchema qw(get_standard_option parse_property_string);
+use PVE::Mapping::Dir ();
+use PVE::RPCEnvironment;
+use PVE::Tools qw(extract_param);
+
+use base qw(PVE::RESTHandler);
+
+__PACKAGE__->register_method ({
+name => 'index',
+path => '',
+method => 'GET',
+# only proxy if we give the 'check-node' parameter
+proxyto_callback => sub {
+   my ($rpcenv, $proxyto, $param) = @_;
+   return $param->{'check-node'} // 'localhost';
+},
+description => "List directory mapping",
+permissions => {
+   description => "Only lists entries where you have 'Mapping.Modify', 
'Mapping.Use' or".
+   " 'Mapping.Audit' permissions on '/mapping/dir/'.",
+   user => 'all',
+},
+parameters => {
+   additionalProperties => 0,
+   properties => {
+   'check-node' => get_standard_option('pve-node', {
+   description => "If given, checks the configurations on the 
given node for ".
+   "correctness, and adds relevant diagnostics for the 
directory to the response.",
+   optional => 1,
+   }),
+   },
+},
+returns => {
+   type => 'array',
+   items => {
+   type => "object",
+   properties => {
+   id => {
+   type => 'string',
+   description => "The logical ID of the mapping."
+   },
+   map => {
+   type => 'array',
+   description => "The entries of the mapping.",
+   items => {
+   type => 'string',
+   description => "A mapping for a node.",
+   },
+   },
+   description => {
+   type => 'string',
+   description => "A description of the logical mapping.",
+   },
+   xattr => {
+   type => 'boolean',
+   description => "Enable support for extended attributes.",
+   optional => 1,
+   },
+   acl => {
+   type => 'boolean',
+   description => "Enable support for posix ACLs (implies 
--xattr).",
+   optional => 1,
+   },
+   checks => {
+   type => "array",
+   optional => 1,
+   description => "A list of checks, only present if 
'check-node' is set.",
+   items => {
+   type => 'object',
+   properties => {
+   severity => {
+   type => "string",
+   enum => ['warning', 'error'],
+   description => "The severity of the error",
+   },
+   message => {
+   type => "string",
+   description => "The message of the error",
+   },
+   },
+   }
+   },
+   },
+   },
+   links

[pve-devel] [PATCH qemu-server v10 5/11] fix #1027: virtio-fs support

[Markus Frank]

add support for sharing directories with a guest vm

virtio-fs needs virtiofsd to be started.
In order to start virtiofsd as a process (despite being a daemon it is does not
run in the background), a double-fork is used.

virtiofsd should close itself together with qemu.

There are the parameters dirid and the optional parameters direct-io, cache and
writeback. Additionally the xattr & acl parameter overwrite the directory
mapping settings for xattr & acl.

The dirid gets mapped to the path on the current node and is also used as a
mount tag (name used to mount the device on the guest).

example config:
```
virtiofs0: foo,direct-io=1,cache=always,acl=1
virtiofs1: dirid=bar,cache=never,xattr=1,writeback=1
```

For information on the optional parameters see the coherent doc patch
and the official gitlab README:
https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/README.md

Also add a permission check for virtiofs directory access.

Signed-off-by: Markus Frank 
---
 PVE/API2/Qemu.pm   |  39 ++-
 PVE/QemuServer.pm  |  19 +++-
 PVE/QemuServer/Makefile|   3 +-
 PVE/QemuServer/Memory.pm   |  34 --
 PVE/QemuServer/Virtiofs.pm | 212 +
 5 files changed, 296 insertions(+), 11 deletions(-)
 create mode 100644 PVE/QemuServer/Virtiofs.pm

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 2a349c8..5d97896 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -695,6 +695,32 @@ my sub check_vm_create_hostpci_perm {
 return 1;
 };
 
+my sub check_dir_perm {
+my ($rpcenv, $authuser, $vmid, $pool, $opt, $value) = @_;
+
+return 1 if $authuser eq 'root@pam';
+
+$rpcenv->check_vm_perm($authuser, $vmid, $pool, ['VM.Config.Disk']);
+
+my $virtiofs = PVE::JSONSchema::parse_property_string('pve-qm-virtiofs', 
$value);
+$rpcenv->check_full($authuser, "/mapping/dir/$virtiofs->{dirid}", 
['Mapping.Use']);
+
+return 1;
+};
+
+my sub check_vm_create_dir_perm {
+my ($rpcenv, $authuser, $vmid, $pool, $param) = @_;
+
+return 1 if $authuser eq 'root@pam';
+
+for my $opt (keys %{$param}) {
+   next if $opt !~ m/^virtiofs\d+$/;
+   check_dir_perm($rpcenv, $authuser, $vmid, $pool, $opt, $param->{$opt});
+}
+
+return 1;
+};
+
 my $check_vm_modify_config_perm = sub {
 my ($rpcenv, $authuser, $vmid, $pool, $key_list) = @_;
 
@@ -705,7 +731,7 @@ my $check_vm_modify_config_perm = sub {
# else, as there the permission can be value dependend
next if PVE::QemuServer::is_valid_drivename($opt);
next if $opt eq 'cdrom';
-   next if $opt =~ m/^(?:unused|serial|usb|hostpci)\d+$/;
+   next if $opt =~ m/^(?:unused|serial|usb|hostpci|virtiofs)\d+$/;
next if $opt eq 'tags';
 
 
@@ -999,6 +1025,7 @@ __PACKAGE__->register_method({
&$check_vm_create_serial_perm($rpcenv, $authuser, $vmid, $pool, 
$param);
check_vm_create_usb_perm($rpcenv, $authuser, $vmid, $pool, $param);
check_vm_create_hostpci_perm($rpcenv, $authuser, $vmid, $pool, 
$param);
+   check_vm_create_dir_perm($rpcenv, $authuser, $vmid, $pool, $param);
 
PVE::QemuServer::check_bridge_access($rpcenv, $authuser, $param);
&$check_cpu_model_access($rpcenv, $authuser, $param);
@@ -1899,6 +1926,10 @@ my $update_vm_api  = sub {
check_hostpci_perm($rpcenv, $authuser, $vmid, undef, $opt, 
$val);
PVE::QemuConfig->add_to_pending_delete($conf, $opt, $force);
PVE::QemuConfig->write_config($vmid, $conf);
+   } elsif ($opt =~ m/^virtiofs\d$/) {
+   check_dir_perm($rpcenv, $authuser, $vmid, undef, $opt, 
$val);
+   PVE::QemuConfig->add_to_pending_delete($conf, $opt, $force);
+   PVE::QemuConfig->write_config($vmid, $conf);
} elsif ($opt eq 'tags') {
assert_tag_permissions($vmid, $val, '', $rpcenv, $authuser);
delete $conf->{$opt};
@@ -1987,6 +2018,12 @@ my $update_vm_api  = sub {
}
check_hostpci_perm($rpcenv, $authuser, $vmid, undef, $opt, 
$param->{$opt});
$conf->{pending}->{$opt} = $param->{$opt};
+   } elsif ($opt =~ m/^virtiofs\d$/) {
+   if (my $oldvalue = $conf->{$opt}) {
+   check_dir_perm($rpcenv, $authuser, $vmid, undef, $opt, 
$oldvalue);
+   }
+   check_dir_perm($rpcenv, $authuser, $vmid, undef, $opt, 
$param->{$opt});
+   $conf->{pending}->{$opt} = $param->{$opt};
} elsif ($opt eq 'tags') {
assert_tag_permissions($vmid, $conf->{$opt}, 
$param->{$opt}, $rpcenv, $authuser);
$conf->{pending}->{$opt} = 
PVE::GuestHelpers::get_unique_tags($param->{$opt});
diff --git a/PVE/QemuServer.pm b/PVE/QemuServ

[pve-devel] [PATCH qemu-server v10 4/11] add virtiofsd as runtime dependency for qemu-server

[Markus Frank]

Signed-off-by: Markus Frank 
---
 debian/control | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/control b/debian/control
index 1301a36..8e4ca7f 100644
--- a/debian/control
+++ b/debian/control
@@ -55,6 +55,7 @@ Depends: dbus,
  socat,
  swtpm,
  swtpm-tools,
+ virtiofsd,
  ${misc:Depends},
  ${perl:Depends},
  ${shlibs:Depends},
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v10 11/11] ui: add options to add virtio-fs to qemu config

[Markus Frank]

Signed-off-by: Markus Frank 
---
 www/manager6/Makefile |   1 +
 www/manager6/Utils.js |   1 +
 www/manager6/qemu/HardwareView.js |  19 +
 www/manager6/qemu/VirtiofsEdit.js | 137 ++
 4 files changed, 158 insertions(+)
 create mode 100644 www/manager6/qemu/VirtiofsEdit.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index cac8cd02..29d676df 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -270,6 +270,7 @@ JSSRC=  
\
qemu/Smbios1Edit.js \
qemu/SystemEdit.js  \
qemu/USBEdit.js \
+   qemu/VirtiofsEdit.js\
sdn/Browser.js  \
sdn/ControllerView.js   \
sdn/Status.js   \
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index f5608944..52ea5589 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1636,6 +1636,7 @@ Ext.define('PVE.Utils', {
serial: 4,
rng: 1,
tpmstate: 1,
+   virtiofs: 10,
 },
 
 // we can have usb6 and up only for specific machine/ostypes
diff --git a/www/manager6/qemu/HardwareView.js 
b/www/manager6/qemu/HardwareView.js
index 672a7e1a..0d4a3984 100644
--- a/www/manager6/qemu/HardwareView.js
+++ b/www/manager6/qemu/HardwareView.js
@@ -309,6 +309,16 @@ Ext.define('PVE.qemu.HardwareView', {
never_delete: !caps.nodes['Sys.Console'],
header: gettext("VirtIO RNG"),
};
+   for (let i = 0; i < PVE.Utils.hardware_counts.virtiofs; i++) {
+   let confid = "virtiofs" + i.toString();
+   rows[confid] = {
+   group: 50,
+   order: i,
+   iconCls: 'folder',
+   editor: 'PVE.qemu.VirtiofsEdit',
+   header: gettext('Virtiofs') + ' (' + confid +')',
+   };
+   }
 
var sorterFn = function(rec1, rec2) {
var v1 = rec1.data.key;
@@ -583,6 +593,7 @@ Ext.define('PVE.qemu.HardwareView', {
const noVMConfigDiskPerm = !caps.vms['VM.Config.Disk'];
const noVMConfigCDROMPerm = !caps.vms['VM.Config.CDROM'];
const noVMConfigCloudinitPerm = !caps.vms['VM.Config.Cloudinit'];
+   const noVMConfigOptionsPerm = !caps.vms['VM.Config.Options'];
 
me.down('#addUsb').setDisabled(noHWPerm || isAtUsbLimit());
me.down('#addPci').setDisabled(noHWPerm || isAtLimit('hostpci'));
@@ -592,6 +603,7 @@ Ext.define('PVE.qemu.HardwareView', {
me.down('#addRng').setDisabled(noSysConsolePerm || 
isAtLimit('rng'));
efidisk_menuitem.setDisabled(noVMConfigDiskPerm || 
isAtLimit('efidisk'));
me.down('#addTpmState').setDisabled(noSysConsolePerm || 
isAtLimit('tpmstate'));
+   me.down('#addVirtiofs').setDisabled(noVMConfigOptionsPerm || 
isAtLimit('virtiofs'));
me.down('#addCloudinitDrive').setDisabled(noVMConfigCDROMPerm || 
noVMConfigCloudinitPerm || hasCloudInit);
 
if (!rec) {
@@ -735,6 +747,13 @@ Ext.define('PVE.qemu.HardwareView', {
disabled: !caps.nodes['Sys.Console'],
handler: editorFactory('RNGEdit'),
},
+   {
+   text: gettext("Virtiofs"),
+   itemId: 'addVirtiofs',
+   iconCls: 'fa fa-folder',
+   disabled: !caps.nodes['Sys.Console'],
+   handler: editorFactory('VirtiofsEdit'),
+   },
],
}),
},
diff --git a/www/manager6/qemu/VirtiofsEdit.js 
b/www/manager6/qemu/VirtiofsEdit.js
new file mode 100644
index ..ec5c69fd
--- /dev/null
+++ b/www/manager6/qemu/VirtiofsEdit.js
@@ -0,0 +1,137 @@
+Ext.define('PVE.qemu.VirtiofsInputPanel', {
+extend: 'Proxmox.panel.InputPanel',
+xtype: 'pveVirtiofsInputPanel',
+onlineHelp: 'qm_virtiofs',
+
+insideWizard: false,
+
+onGetValues: function(values) {
+   var me = this;
+   var confid = me.confid;
+   var params = {};
+   delete values.delete;
+   params[confid] = PVE.Parser.printPropertyString(values, 'dirid');
+   return params;
+},
+
+setSharedfiles: function(confid, data) {
+   var me = this;
+   me.confid = confid;
+   me.virtiofs = data;
+   me.setValues(me.virtiofs);
+},
+initComponent: function() {
+   let me = this;
+
+   me.nodename = me.pveSelNode.data.node;
+   if (!me.nodename) {
+   throw "no node name specified";
+   }
+   me.items = [
+   {
+   xtype: 'pveDirMapS

[pve-devel] [PATCH guest-common v10 2/11] add dir mapping section config

[Markus Frank]

Adds a config file for directories by using a 'map' property string for
each node mapping.

Next to node & path, there is the optional submounts parameter in the
map property string that is used to announce other mounted file systems
in the specified directory.

Additionally there are the default settings for xattr & acl.

example config:
```
some-dir-id
map node=node1,path=/mnt/share/,submounts=1
map node=node2,path=/mnt/share/,
xattr 1
acl 1
```

Signed-off-by: Markus Frank 
---
 src/Makefile   |   1 +
 src/PVE/Mapping/Dir.pm | 205 +
 2 files changed, 206 insertions(+)
 create mode 100644 src/PVE/Mapping/Dir.pm

diff --git a/src/Makefile b/src/Makefile
index cbc40c1..030e7f7 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -15,6 +15,7 @@ install: PVE
install -m 0644 PVE/StorageTunnel.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/Tunnel.pm ${PERL5DIR}/PVE/
install -d ${PERL5DIR}/PVE/Mapping
+   install -m 0644 PVE/Mapping/Dir.pm ${PERL5DIR}/PVE/Mapping/
install -m 0644 PVE/Mapping/PCI.pm ${PERL5DIR}/PVE/Mapping/
install -m 0644 PVE/Mapping/USB.pm ${PERL5DIR}/PVE/Mapping/
install -d ${PERL5DIR}/PVE/VZDump
diff --git a/src/PVE/Mapping/Dir.pm b/src/PVE/Mapping/Dir.pm
new file mode 100644
index 000..8f131c2
--- /dev/null
+++ b/src/PVE/Mapping/Dir.pm
@@ -0,0 +1,205 @@
+package PVE::Mapping::Dir;
+
+use strict;
+use warnings;
+
+use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_lock_file 
cfs_write_file);
+use PVE::INotify;
+use PVE::JSONSchema qw(get_standard_option parse_property_string);
+use PVE::SectionConfig;
+use PVE::Storage::Plugin;
+
+use base qw(PVE::SectionConfig);
+
+my $FILENAME = 'mapping/dir.cfg';
+
+cfs_register_file($FILENAME,
+  sub { __PACKAGE__->parse_config(@_); },
+  sub { __PACKAGE__->write_config(@_); });
+
+
+# so we don't have to repeat the type every time
+sub parse_section_header {
+my ($class, $line) = @_;
+
+if ($line =~ m/^(\S+)\s*$/) {
+   my $id = $1;
+   my $errmsg = undef; # set if you want to skip whole section
+   eval { PVE::JSONSchema::pve_verify_configid($id) };
+   $errmsg = $@ if $@;
+   my $config = {}; # to return additional attributes
+   return ('dir', $id, $errmsg, $config);
+}
+return undef;
+}
+
+sub format_section_header {
+my ($class, $type, $sectionId, $scfg, $done_hash) = @_;
+
+return "$sectionId\n";
+}
+
+sub type {
+return 'dir';
+}
+
+my $map_fmt = {
+node => get_standard_option('pve-node'),
+path => {
+   description => "Absolute directory path that should be shared with the 
guest.",
+   type => 'string',
+   format => 'pve-storage-path',
+},
+submounts => {
+   type => 'boolean',
+   description => "Announce that the directory contains other mounted"
+   ." file systems. If this is not set and multiple file systems are"
+   ." mounted, the guest may encounter duplicates due to file system"
+   ." specific inode IDs.",
+   optional => 1,
+   default => 0,
+},
+description => {
+   description => "Description of the node specific directory.",
+   type => 'string',
+   optional => 1,
+   maxLength => 4096,
+},
+};
+
+my $defaultData = {
+propertyList => {
+   id => {
+   type => 'string',
+   description => "The ID of the directory",
+   format => 'pve-configid',
+   },
+   description => {
+   description => "Description of the directory",
+   type => 'string',
+   optional => 1,
+   maxLength => 4096,
+   },
+   map => {
+   type => 'array',
+   description => 'A list of maps for the cluster nodes.',
+   optional => 1,
+   items => {
+   type => 'string',
+   format => $map_fmt,
+   },
+   },
+   xattr => {
+   type => 'boolean',
+   description => "Enable support for extended attributes."
+   ." If not supported by Guest OS or file system, this option is"
+   ." simply ignored.",
+   optional => 1,
+   default => 0,
+   },
+   acl => {
+   type => 'boolean',
+   description => "Enable support for POSIX ACLs (implies --xattr)."
+   ." The guest OS has to support ACLs. When used in a directory"
+   ." with a file system without ACL support, the ACLs are 
ignored.",
+   optional => 1,
+   default => 0,
+   },
+},
+};
+
+sub private {
+return $defaultData;
+}
+
+sub map_fmt {
+r

[pve-devel] [PATCH cluster v10 1/11] add mapping/dir.cfg for resource mapping

[Markus Frank]

Add it to both the perl side (PVE/Cluster.pm) and pmxcfs side
(status.c).
This dir.cfg is used to map directory IDs to paths on selected hosts.

Signed-off-by: Markus Frank 
Reviewed-by: Fiona Ebner 
---
 src/PVE/Cluster.pm  | 1 +
 src/pmxcfs/status.c | 1 +
 2 files changed, 2 insertions(+)

diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index f899dbe..6b775f8 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -82,6 +82,7 @@ my $observed = {
 'sdn/.running-config' => 1,
 'virtual-guest/cpu-models.conf' => 1,
 'virtual-guest/profiles.cfg' => 1,
+'mapping/dir.cfg' => 1,
 'mapping/pci.cfg' => 1,
 'mapping/usb.cfg' => 1,
 };
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index dc44464..17cbf61 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -112,6 +112,7 @@ static memdb_change_t memdb_change_array[] = {
{ .path = "virtual-guest/cpu-models.conf" },
{ .path = "virtual-guest/profiles.cfg" },
{ .path = "firewall/cluster.fw" },
+   { .path = "mapping/dir.cfg" },
{ .path = "mapping/pci.cfg" },
{ .path = "mapping/usb.cfg" },
 };
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [ccp4bb] AlphaFold3 Transparency and Reproducibility

[Frank von Delft]

What we really need is a tweet from Nature, declaring their 
embarrassment at having been suckered into becoming DeepMind's 
advertising arm.



On 14/05/2024 01:56, Paul Adams wrote:


The letter may have had (or helped have) an impact already:

On X today from Pushmeet Kohli @ DeepMind

"We love the excitement & results from the community on AlphaFold 3 
and are doubling the AF Server daily job limit to 20. Happy to also 
share that we're working on releasing the AF3 model (incl weights) for 
academic use, which doesn’t depend on our research infra, within 6 
months".



On May 13, 2024, at 11:11 AM, Wankowicz, Stephanie 
 wrote:


If it does what it claims – incredibly impressive. The issue we have 
is there is no way to verify or validate the claims it is making.


The letter is a call and start of a conversation about how the 
ever-changing landscape of communication and publication of science 
should be.


-Stephanie Wankowicz

*From:*CCP4 bulletin board  on behalf of 
Krieger, James M 

*Sent:*Monday, May 13, 2024 10:22 AM
*To:*CCP4BB@JISCMAIL.AC.UK
*Subject:*Re: [ccp4bb] Fwd: AlphaFold3 Transparency and Reproducibility
This Message Is From an External Sender
This message came from outside your organization.
It definitely is impressive but it also has clear limitations

On 13 May 2024, at 15:22, Sylvia Fanucchi 
 wrote:





You don't often get email from 
d0c4e77ae410-dmarc-requ...@jiscmail.ac.uk.Learn why this is 
important 




Is it just me who is really impressed by it? Am I missing something?

GetOutlook for Android 



*From:*CCP4 bulletin board  on behalf of 
Rafael Marques 

*Sent:*Monday, May 13, 2024 3:13:16 PM
*To:*CCP4BB@JISCMAIL.AC.UK 
*Subject:*Re: [ccp4bb] Fwd: AlphaFold3 Transparency and Reproducibility
I tried it yesterday and I was really shocked by how fast it is. 
When I was preparing to submit my second job, the first one was 
already finished, which made me think that I was definitely doing 
something wrong. Probably I was...


Best wishes

Rafael Marques


On 13 May 2024 09:53, Harry Powell 
<193323b1e616-dmarc-requ...@jiscmail.ac.uk> wrote:


Hi folks

This arrived in my inbox this morning, and I believe that it may
provoke some discussion…

Wikipedia tells me: στέργει γὰρ οὐδεὶς ἄγγελον κακῶν ἐπῶν

Best wishes!

Harry

>    From: Stephanie Wankowicz 
>    Sent: Saturday, May 11, 2024 3:31 PM
>    To: James Fraser ; Pedro Beltrao
; Benjamin Cravatt
; Roland Dunbrack
; Anthony Gitter
; Kresten Lindorff-Larsen
; Sergey Ovchinnikov ; Polizzi,
Nicholas F. ; Brian Shoichet

>    Subject: AlphaFold3 Transparency and Reproducibility
>
>
>    This email from mullane.stepha...@gmail.com originates from
outside Imperial. Do not click on links and attachments unless
you recognise the sender. If you trust the sender, add them to
your safe senders list


to
disable email stamping for this address.
>
>    Hello,
>
>
>    As many of you, we were incredibly disappointed with the
lack of code or even executables accompanying the publication of
AlphaFold3 in Nature. AlphaFold3 was released without the means
to test and use the software in a high-throughput manner. This
does not align with the principles of scientific research, which
rely on the ability of the community to evaluate, use, and build
upon existing work.
>
>
>
>    We have written a letter, which will be posted on Zenodo
and submitted as a Letter to the Editor in the coming days.
>
>
>
>    Please see the entire letter here.



If
you want to endorse this letter, please fill out your name,
affiliation, and email in the form.
>
>
>
>    

[PATCH] drm/amdgpu/mes: use mc address for wptr in add queue packet

[Min, Frank]

[AMD Official Use Only - AMD Internal Distribution Only]

From: Frank Min 

use mc address for wptr in add queue packet

Signed-off-by: Frank Min 
---
 drivers/gpu/drm/amd/amdgpu/mes_v12_0.c | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c 
b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
index 5519655fd70a..6256b21884ee 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
@@ -267,11 +267,7 @@ static int mes_v12_0_add_hw_queue(struct amdgpu_mes *mes,
mes_add_queue_pkt.doorbell_offset = input->doorbell_offset;
mes_add_queue_pkt.mqd_addr = input->mqd_addr;

-   if (((adev->mes.sched_version & AMDGPU_MES_API_VERSION_MASK) >>
-   AMDGPU_MES_API_VERSION_SHIFT) >= 2)
-   mes_add_queue_pkt.wptr_addr = input->wptr_mc_addr;
-   else
-   mes_add_queue_pkt.wptr_addr = input->wptr_addr;
+   mes_add_queue_pkt.wptr_addr = input->wptr_mc_addr;

mes_add_queue_pkt.queue_type =
convert_to_mes_queue_type(input->queue_type);
--
2.34.1

[kaddressbook] [Bug 58706] custom types for addresses and phone numbers

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=58706

Peer Frank  changed:

   What|Removed |Added

 CC||peer.fr...@web.de

-- 
You are receiving this mail because:
You are watching all bug changes.

[kaddressbook] [Bug 478637] Add an additional field "Date of death"

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=478637

Peer Frank  changed:

   What|Removed |Added

 CC||peer.fr...@web.de

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kaddressbook] [Bug 478637] Add an additional field "Date of death"

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=478637

Peer Frank  changed:

   What|Removed |Added

 CC||peer.fr...@web.de

-- 
You are receiving this mail because:
You are watching all bug changes.

[kaddressbook] [Bug 486489] edit contact dialog: Saving an (un)modified contact, always trigger an 'location modified' confirmation

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=486489

--- Comment #1 from Peer Frank  ---
yes, that's annoying

-- 
You are receiving this mail because:
You are watching all bug changes.

[kaddressbook] [Bug 486489] edit contact dialog: Saving an (un)modified contact, always trigger an 'location modified' confirmation

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=486489

--- Comment #1 from Peer Frank  ---
yes, that's annoying

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kaddressbook] [Bug 486489] edit contact dialog: Saving an (un)modified contact, always trigger an 'location modified' confirmation

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=486489

Peer Frank  changed:

   What|Removed |Added

 CC||peer.fr...@web.de
   Platform|Neon|openSUSE

-- 
You are receiving this mail because:
You are the assignee for the bug.

[kaddressbook] [Bug 486489] edit contact dialog: Saving an (un)modified contact, always trigger an 'location modified' confirmation

[Peer Frank]

https://bugs.kde.org/show_bug.cgi?id=486489

Peer Frank  changed:

   What|Removed |Added

 CC||peer.fr...@web.de
   Platform|Neon|openSUSE

-- 
You are receiving this mail because:
You are watching all bug changes.

[i18n] [Bug 474572] Plasma5-pk-updates with mixed languages

[Frank Steinmetzger]

https://bugs.kde.org/show_bug.cgi?id=474572

Frank Steinmetzger  changed:

   What|Removed |Added

 CC||dev-...@felsenfleischer.de

--- Comment #7 from Frank Steinmetzger  ---
(In reply to Frank Kruger from comment #5)

> The issue is solved, if ":" is omitted in the translation for "Check for
> Updates" in plasma_applet_org.kde.plasma.pkupdates.po, i.e., "Auf
> Aktualisierungen prüfen"

I fixed that one now, so it should trickle down into packages over time.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Bug 2056173] Re: dotnet 8/s390x (8.0.2 runtime / 8.0.102 SDK) feedback

[Frank Heimes]

** Also affects: dotnet8 (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: dotnet8 (Ubuntu)
   Status: New => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2056173

Title:
  dotnet 8/s390x (8.0.2 runtime / 8.0.102 SDK) feedback

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2056173/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: Smaller buildroot for Perl packages

[Frank Ch. Eigler]

Hi -

> > OK, build-time dependency changes may not need the side tag but do
> > need a spec update to prevent a FTBFS at next build.
> 
> Only those packages that actually need dtrace would require changes. Such
> changes could land gradually as needed.

They'd have to land at the next respin of each of those packages.

- FChE
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

getting debian derivate into a debian-edu network

[Frank Weißer]

Hi,

I've been looking for a brief description on how to get a workstation 
with a debian derivate (here mx-/av-linux) to act like a normal 
debianedu workstation with home directories, user login to ldap, etc.


As I didn't find anything, started with installing tasksel and
education-tasks. Then started tasksel and chose 'Debian Edu networked 
workstation packages'.

for ldap server I chose 'ldap://10.0.2.2/'
defaultsearchbase "dc=skole,dc=skolelinux,dc=no
real for Kerberos 'goKrbRealm'
kerberos server for realm '10.0.2.2'
administration server for realm '10.0.2.2'

Am I right so far?

next should be name resolving services, but which to choose?

Any hints on this an on the upcoming next parameters?

Would be nice, if defaults matching for tjener were offered.

Kind regards

Re: RTEMS Project Repos in GitLab

[Frank Kühndel]

Hello Chris,

I just did not managed to find git://git.rtems.org/rtems_waf.git on 
gitlab. Has been moved too or will it stay were it was?


Thanks,
Frank

On 5/1/24 08:46, Chris Johns wrote:

Hi RTEMS Community.

This email explains the RTEMS Project and Repos in GitLab.

GitLab is will be open and accounts can be created from 12PM EST / 5PM GM.

Note, GitLab failed to automatically identify the RTOS/RTEMS licence and it
incorrectly states it is Apache 2.0. It is not and no licences have changed. An
issue is open to sort this out.

GitLab lets you have repositories structured as projects. We have a top level
namespace called RTEMS and our repositories reside under this namespace. Having
everything under a single namespace lets us use a lot of cool GitLab features.
The link ishttps://gitlab.rtems.org/rtems.

We have some other namespaces alongside RTEMS such as Administration, Approvers
and Architecture. The projects in the top level namespace are place holders for
the groups we use across GitHub and groups let us control what users can do.
The Administration project is used to run GitLab and the servers we have at OSL
OSU. If you have found a bug or problem please raise an issue. If you have a
feature or a good idea please raise an issue. If you know an answer to an issue
that is open please answer.

The Architecture project lets us define groups of users who can own and approve
architecture specific parts of RTEMS. We have this at the top so architecture
owners can manage CI runners.

The top level RTEMS Project contains the following project groups:

  1. Documentation - Our documentation.

  2. Packages - The various packages we can build for RTEMS. Think of them
 as 3rd party libraries

  3. Pre-Qualification - The pre-qualification projects based on the
 recent ESA funded work

  4. RTOS - The real-time kernel, its examples and the release scripts

  5. Tools - Our ecosystem tools such as the RSB, RTEMS Tools and the
 SIS simulator

  6. Programs - A place to organise activities similar or related to
 Google Summer of Code

Main Branch

We have chosen this time to move from a master branch to main on all our repos.
Leaving the git.rtems.org repos open while we worked on GitLab meant the project
could continue to function as we deployed GitLab however we have had to make
commits in the GitLab repos to add files like CODEOWNERS and to test the
platform. The simplest solution for us is to roll any commits to git.rtems.org
into the GitLab repos before we go live. If you have maintained any waiting to
be merged on local branches in your repo you can rebase to the main branch. The
following is an example of how to switch an RTEMS git.rtems.org repo you use to
track RTEMS to RTEMS GitLab:

  1. The procedure assumes your local master branch matches the git.rtems.org
 master branch and any changes you have are on local branches.

  2. Set the remote  origin to GitLab:
  git remote set-url 
originssh://g...@gitlab.rtems.org:/rtems/rtos/rtems.git

  3. Fetch the GitLab repo:
  git fetch

  4. Check out the GitLab main branch:
  git checkout main

  5. We recommend you delete your local master branch:
  git branch -D master

The following is an example of how to switch an RTEMS git.rtems.org repo you
have changed in you wish to post for review and merging to RTEMS GitLab:

  1. The procedure assumes your local master branch matches the git.rtems.org
 master branch and any changes you have are on local branches.

  2. Fork the GitLab repo to your user account. Open the project repo in GitLab
 and click Fork, located top right. You will need to select a namespace
 and it needs to be the one based on your user name. Select the namespace
 drop down in the Project URL field and scroll down or enter your user name
 and click on the entry.

  3. Set the remote  origin to GitLab:
  git remote set-url originssh://g...@gitlab.rtems.org:/chris/rtems.git
 Replace my user name with yours.

  4. Fetch the GitLab repo:
  git fetch

  5. Check out the GitLab main branch:
  git checkout main

  6. We recommend you delete your local master branch:
  git branch -D master

Merge Request Approvers

Every merge request in RTEMS requires two approvers. Approvers are organized as
members of subgroups within the top-level Approvers group
https://gitlab.rtems.org/approvers.

We have added initial CODEOWNERS files in GitLab that details which subgroups
own and approve merge requests. There are some members of the community who need
to be added to specific subgroups but we cannot add them until they have
accounts. If you want to be a code owner and approver please raise an
Administration issue. Anyone in the subgroup of “General Approver” is able to be
one of the approvers of  merge requests. However, at least one approver must
belong to a subgroup that is a code owner for a specific file or directory.
Self-approval by the author of the merge request

Re: Smaller buildroot for Perl packages

[Frank Ch. Eigler]

Hi -

> > > > I also did a test rebuild of all packages directly build-requiring
> > > > systemtap-sdt-devel and identified these packages that really need the
> > > > dtrace script: [...]
> > (The logistic challenge there will be side-tag rebuilding all those
> > after a systemtap subrpm split.)
> 
> I don't understand why that would be necessary. Could you please explain why
> do you believe it would?

OK, build-time dependency changes may not need the side tag but do
need a spec update to prevent a FTBFS at next build.

- FChE
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: [PATCH] Support nullglob in profile.sh.in

[Frank Ch. Eigler]

Hi -

> > Don't block on stdin when /etc/debuginfod/*.certpath expands to nothing.
> > Signed-off-by: Andreas Schwab 

Thanks, committed as obvious.


> I slightly prefer "cat  but this variant is also fine.

I see your point, but the other five cases in the profile.* use this
formulation, so for simpleminded consistency, went with the above.

- FChE

[PATCH] drm/amdgpu: fix getting vram info for gfx12

[Min, Frank]

[AMD Official Use Only - AMD Internal Distribution Only]

From: Frank Min 

gfx12 query video mem channel/type/width from umc_info of atom list, so fix it 
accordingly.

Signed-off-by: Frank Min 
---
 .../gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c  | 263 ++
 1 file changed, 148 insertions(+), 115 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
index a6d64bdbbb14..6fe84151332e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
@@ -289,7 +289,6 @@ static int convert_atom_mem_type_to_vram_type(struct 
amdgpu_device *adev,
return vram_type;
 }

-
 int
 amdgpu_atomfirmware_get_vram_info(struct amdgpu_device *adev,
  int *vram_width, int *vram_type,
@@ -300,6 +299,7 @@ amdgpu_atomfirmware_get_vram_info(struct amdgpu_device 
*adev,
u16 data_offset, size;
union igp_info *igp_info;
union vram_info *vram_info;
+   union umc_info *umc_info;
union vram_module *vram_module;
u8 frev, crev;
u8 mem_type;
@@ -311,10 +311,16 @@ amdgpu_atomfirmware_get_vram_info(struct amdgpu_device 
*adev,
if (adev->flags & AMD_IS_APU)
index = 
get_index_into_master_table(atom_master_list_of_data_tables_v2_1,
integratedsysteminfo);
-   else
-   index = 
get_index_into_master_table(atom_master_list_of_data_tables_v2_1,
-   vram_info);
-
+   else {
+   switch (amdgpu_ip_version(adev, GC_HWIP, 0)) {
+   case IP_VERSION(12, 0, 0):
+   case IP_VERSION(12, 0, 1):
+   index = 
get_index_into_master_table(atom_master_list_of_data_tables_v2_1, umc_info);
+   break;
+   default:
+   index = 
get_index_into_master_table(atom_master_list_of_data_tables_v2_1, vram_info);
+   }
+   }
if (amdgpu_atom_parse_data_header(mode_info->atom_context,
  index, ,
  , , _offset)) {
@@ -368,123 +374,150 @@ amdgpu_atomfirmware_get_vram_info(struct amdgpu_device 
*adev,
return -EINVAL;
}
} else {
-   vram_info = (union vram_info *)
-   (mode_info->atom_context->bios + data_offset);
-   module_id = (RREG32(adev->bios_scratch_reg_offset + 4) 
& 0x00ff) >> 16;
-   if (frev == 3) {
-   switch (crev) {
-   /* v30 */
-   case 0:
-   vram_module = (union vram_module 
*)vram_info->v30.vram_module;
-   mem_vendor = 
(vram_module->v30.dram_vendor_id) & 0xF;
-   if (vram_vendor)
-   *vram_vendor = mem_vendor;
-   mem_type = vram_info->v30.memory_type;
-   if (vram_type)
-   *vram_type = 
convert_atom_mem_type_to_vram_type(adev, mem_type);
-   mem_channel_number = 
vram_info->v30.channel_num;
-   mem_channel_width = 
vram_info->v30.channel_width;
-   if (vram_width)
-   *vram_width = 
mem_channel_number * (1 << mem_channel_width);
-   break;
-   default:
-   return -EINVAL;
-   }
-   } else if (frev == 2) {
-   switch (crev) {
-   /* v23 */
-   case 3:
-   if (module_id > 
vram_info->v23.vram_module_num)
-   module_id = 0;
-   vram_module = (union vram_module 
*)vram_info->v23.vram_module;
-   while (i < module_id) {
-   vram_module = (union 
vram_module *)
-   ((u8 *)vram_module + 
vram_module->v9.vram_module_size);
-   i++;
-   }
-   mem_type = vram_module->v9.memory_type;
-   if (vram_type)
-  

Re: [DBWG] why can't i change role object?

[Frank Habicht]

Hi Yogesh,

no, i think my expectation/requirement is different.

role object A is created for a specific purpose, say the "Hostmasters" 
at my employer.


we want to be adding new person contacts to the role object as new real 
people take up this role/join that team, and we want to remove persons 
from admin-c and tech-c as they leave the position or the company. I 
believe this flexibility - without changes in the objects referencing 
the role object - is the main benefit of role objects.


Specifically, i believe i want to use role objects in admin-c and tech-c 
of ORG objects.


however, the error message told me I can not modify the role object.
that left me confused.


Regards,
Frank



On 13/05/2024 10:54, Yogesh Chadee via DBWG wrote:

Good morning Mr. Habicht.

I wish to understand your expectations on this matter. I am thinking you 
want effect this type of change:


"nic-hdl X had role A, now nic-hdl X has role B."

Am I correct in my assumption?


Regards,

Yogesh


On 10/05/2024 15:51, Frank Habicht wrote:

Hi all,

I hope/trust that's not a question for "Support"...

I tried to update/modify a role object, and got this (sorry for the 
line breaks):



***Error:   Person/Role name cannot be changed automatically. Please 
create
    another Person/Role object and modify any references to 
the old

    object, then delete the old object



I can't see a reason why updates to role projects should not be allowed.
I even think that's the main purpose

So, am I missing something?


Thanks,
Frank



PS: AfriNIC staff:
I got "Internal software error" when i wanted to delete
SNHT5-AFRINIC
less than 1 minute after creating it

wanted to delete it with the line
delete: remove
added

according to
delete: 
under section "2.2.4 Deleting an object"
in
https://afrinic.net/press/197-database-afrinic-database-reference-manual-

deleting without the comment worked.

Can that section in that manual be reviewed?
If google sent me to an outdated document, can we remove it?


Thanks,
Frank

___
DBWG mailing list
DBWG@afrinic.net
https://lists.afrinic.net/mailman/listinfo/dbwg


___
DBWG mailing list
DBWG@afrinic.net
https://lists.afrinic.net/mailman/listinfo/dbwg


___
DBWG mailing list
DBWG@afrinic.net
https://lists.afrinic.net/mailman/listinfo/dbwg

[Bug 2065579] Re: [UBUNTU 22.04] OS guest boot issues on 9p filesystem

[Frank Heimes]

** Package changed: linux (Ubuntu) => qemu (Ubuntu)

** Also affects: ubuntu-z-systems
   Importance: Undecided
   Status: New

** Changed in: ubuntu-z-systems
 Assignee: (unassigned) => Skipper Bug Screeners (skipper-screen-team)

** Changed in: qemu (Ubuntu)
 Assignee: Skipper Bug Screeners (skipper-screen-team) => (unassigned)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2065579

Title:
  [UBUNTU 22.04] OS guest boot issues on 9p filesystem

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/2065579/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[PATCH] drm/amdgpu: add initial value for gfx12 AGP aperture

[Min, Frank]

[AMD Official Use Only - AMD Internal Distribution Only]

From: Frank Min 

add initial value for gfx12 AGP aperture

Signed-off-by: Frank Min 
---
 drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c 
b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
index 34264a33dcdf..b876300bb9f6 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
@@ -622,6 +622,7 @@ static void gmc_v12_0_vram_gtt_location(struct 
amdgpu_device *adev,

base = adev->mmhub.funcs->get_fb_location(adev);

+   amdgpu_gmc_set_agp_default(adev, mc);
amdgpu_gmc_vram_location(adev, >gmc, base);
amdgpu_gmc_gart_location(adev, mc, AMDGPU_GART_PLACEMENT_LOW);
if (!amdgpu_sriov_vf(adev) && (amdgpu_agp == 1))
--
2.34.1

[posted] Posted (#73619, Sinclair) !

[Roger Frank]

One good turn, by Bertrand W. Sinclair73619
  [Author: Bertrand W. Sinclair]
  [Link: https://www.gutenberg.org/ebooks/73619]
  note: link will be active within two hours of posting.

Thanks to Roger Frank and Sue Clark

[posted] Posted (#73618, Davis) !

[Roger Frank]

Splashes of red, by J. Frank Davis73618
  [Author: J. Frank Davis]
  [Link: https://www.gutenberg.org/ebooks/73618]
  note: link will be active within two hours of posting.

Thanks to Roger Frank and Sue Clark

Re: How to change fail2ban timeout during reboot in F40?

[Frank Bures]

On 2024-05-12 16:11, Anthony Messina wrote:

Of course, you'll want to set something suitable for your system's needs


~]# cat /etc/systemd/system/fail2ban.service.d/timeout.conf
[Service]
TimeoutStopSec=5min



Thanks
Frank

--


--
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

How to change fail2ban timeout during reboot in F40?

[Frank Bures]

Hi,

I was a target of an attack resulting in 1600 banned IP address by fail2ban.

When I reboot, f2b tries to unban those IPs but it ultimately times out and 
a large number of IPs becomes orphaned.  They stay in firewalld, but f2b 
does not know about them anymore.


And indeed

root@ryzen:/etc/systemd/system# systemctl show fail2ban.service -p 
TimeoutStopUSec


TimeoutStopUSec=45s

How do I increase the 45s timeout to something more suitable?  I was not 
able to find where in the system is that value set.


Thanks
Frank

--


--
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

[kget] [Bug 486787] Paused downloads cannot be resumed

[Frank Kruger]

https://bugs.kde.org/show_bug.cgi?id=486787

Frank Kruger  changed:

   What|Removed |Added

 CC||fkrue...@mailbox.org

--- Comment #1 from Frank Kruger  ---
I am not sure whether the culprit is the same, but KGet (24.0.2) is always
stalled at a certain percentage (12 or 24 %) and does not resume at all (worked
fine with Plasma 5).

OS: openSUSE Tumbleweed 20240509
KDE Plasma Version: 6.0.4
KDE Frameworks Version: 6.1.0
Qt Version: 6.7.0

-- 
You are receiving this mail because:
You are watching all bug changes.

Re: [users@httpd] Multi site SSL problems

[Frank Gingras]

On Fri, May 10, 2024 at 5:53 PM Tatsuki Makino 
wrote:

> Hello.
>
> By the way, do you have the setting enabled to use the Host header used to
> switch NameVirtualHost during TLS negotiation?
> I don't know how to do that since the Japanese documentation is rarely
> updated :)
> Were those things implemented?
>
> Regards.
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
> For additional commands, e-mail: users-h...@httpd.apache.org
>
>
Tatsuki,

You're thinking of SNI, and it works out of the box with OpenSSL 0.9.8f or
later, and with NameVirtualHost *:443.

So, again, I highly recommend using *:PORT to define all your vhosts,
unless you know exactly what you are doing.

Re: [users@httpd] Multi site SSL problems

[Frank Gingras]

On Fri, May 10, 2024 at 4:10 PM John  wrote:

> On Fri, 2024-05-10 at 15:48 -0400, Sean Conner wrote:
> > It was thus said that the Great Chris me once stated:
> > > I set up each entry with  but when I do that, the
> > > second site will complain that the cert is for site1. So if I go to
> > > site2.com, I get a browser error that the cert is for site1. It will
> show
> > > me the content for site1.
> >
> >   On my development server, I have the following:
> >
> > 
> >   ServerName  playground.roswell.area51
> >   SSLEngine   on
> >   SSLCertificateFile  /home/spc/web/playground/cert.pem
> >   SSLCertificateKeyFile   /home/spc/web/playground/key.pem
> >   ...
> > 
> >
> > 
> >   ServerName  wiki.roswell.area51
> >   SSLEngine   on
> >   SSLCertificateFile  /home/spc/web/wiki/cert.pem
> >   SSLCertificateKeyFile   /home/spc/web/wiki/key.pem
> >   ...
> > 
> >
> > > I am not sure how to do this part:
> > > Do not use the 2.2 authz directives (Allow/Deny/Order) and use Require
> instead
> > > I am running Apache 2.2, does it still apply?
> > > It does not look like mod_access_compat is listed under mods-enabled
> >
> >   That I don't remember as I've been running Apache 2.4 for a couple of
> > years now.
> >
> >   -spc
> >
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
> > For additional commands, e-mail: users-h...@httpd.apache.org
> >
> Typo in the 2nd virtual host "1932.168.1.10:"  probably should be
> "192.168.1.10"
>
> John
> ==
>
> -
> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
> For additional commands, e-mail: users-h...@httpd.apache.org
>
>
Show the apachectl -S output, and each vhost.  Make sure that every single
:443 vhost has SSLEngine on and SSLCertificateFile set.

Re: [dnsdist] Question about local override

[Frank @ kiwazo.be via dnsdist]

Dnsdist would always need an authoritative server to server domains (note that 
you can distill dns replies in dnsdist, it really shouldn't be used for your 
use case).


The backend doesn't need to be PowerDNS: dnsdist is just used to send traffic 
to a different backend based on the parameters you decide (eg source ip)

Frank


> On 10 May 2024, at 22:47, Rory Toma  wrote:
> 
> Thanks, that does help, but I was hoping I could do it all in dnsdist w/o 
> implementing a pdns backend yet.
> 
> -Original Message-
> From: Frank Louwers  
> Sent: Friday, May 10, 2024 4:31 PM
> To: Rory Toma ; dnsdist mailing list 
> 
> Subject: Re: [dnsdist] Question about local override
> 
> CAUTION: This email is originated from outside of the organization. Do not 
> click links or open attachments unless you recognize the sender and know the 
> content is safe.
> 
> 
> Hi Rory,
> 
> Does this help:
> https://www.frank.be/implementing-bind-views-with-powerdns/
> 
> 
> Regards,
> 
> Frank
> 
> 
> Frank Louwers
> PowerDNS Certified Consultant @ Kiwazo.be
> 
>> On 10 May 2024, at 22:21, Rory Toma via dnsdist 
>>  wrote:
>> 
>> My current setup uses dns views (which I abhor).> So, depending on the 
>> subnet the request comes from, certain domains  get different answers.
>> I’d like to use dnsdist, but I’m having difficulty in implementing  
>> domain overrides, and I haven’t been able to find a good, clean  
>> example.
>> So lets say I have a domain view1.company.com that resolves normally.
>> On my dnsdist server, I’d like to override this locally, and anyone  
>> that talks to that servers gets the overrides. What’s the best way to  
>> do this?
>> thx
>> ___
>> dnsdist mailing list
>> dnsdist@mailman.powerdns.com
>> https://mailman.powerdns.com/mailman/listinfo/dnsdist
> 
> 

___
dnsdist mailing list
dnsdist@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/dnsdist

Re: Smaller buildroot for Perl packages

[Frank Ch. Eigler]

Hi -


> [...]
> > My idea is to split systemtap-sdt-devel into two packages: one with all the
> > content but without the python script (/usr/bin/dtrace) and a new one
> > containing only the mentioned script.

No objection here.


> > [...]
> > I also did a test rebuild of all packages directly build-requiring
> > systemtap-sdt-devel and identified these packages that really need the
> > dtrace script: glib2, sssd, qemu, python2.7, postgresql15, postgresql16,
> > perl, php, mariadb10.11, and libvirt. Those would newly depend on a new
> > package where we move the script to.

(The logistic challenge there will be side-tag rebuilding all those
after a systemtap subrpm split.)

How much time did excluding the python bits from the perl buildroot
actually save during your tests? 


> Unanswered question is run-time dependencies. There might be packages which
> run-require systemtap-sdt-devel because of dtrace executable:
> 
> # dnf -q -C repoquery --whatrequires systemtap-sdt-devel
> lttng-ust-devel-0:2.13.8-1.fc41.x86_64
> perl-devel-4:5.38.2-507.fc41.x86_64
> systemtap-testsuite-0:5.1~pre17062192g5fd8daba-1.fc40.x86_64

By default, these packages could inherit dependencies on both of the
split subrpms, until their respective packagers decide to analyze and
narrow it further down.

- FChE
--
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: [sr #111044] autoconf should assert existence of all subsidiary tools at startup

[Frank Ch. Eigler]

Hi -

> Frank, do you have details of those real platforms with missing diff?
> As I wrote in <https://savannah.gnu.org/support/index.php?111044>, it's
> difficult to implement this kind of sanity check without access to an
> environment that *fails* this kind of sanity check.  I could set one up
> myself but I wouldn't know if I had picked the right things to leave out.

Not a live system I use day-to-day.  But I mean I can remove
"diffutils" from my Fedora box, and see it break.

I think the point was not so much that normal & typical developer
systems may fail.  It's that unforseen restricted machines (maybe
running in minimal fashionably-"secure" buildroots) could encounter
these problems, and then QUIETLY FAIL by misconfiguring tools.

Thus the idea was just to do lightweight & obvious sanity checking on
the environment at configure startup.  e.g.:

diff /dev/null /dev/null || exit 1


- FChE

Re: [RFC] pci: mediatek: add PCIe controller support for Filogic

[Frank Wunderlich]

Am 10. Mai 2024 15:08:19 MESZ schrieb Tom Rini :
>On Fri, May 10, 2024 at 01:57:26PM +0200, Frank Wunderlich wrote:
>
>> a gentle ping...any comments?
>
>Seems fine?

> Can we switch to OF_UPSTREAM first however?

Pulling mediatek devicetrees from linux will afaik break network in uboot for 
mt7622,7623,mt7986 and mt7988 bananapi boards because switch (mt753x) binding 
is different to linux.

regards Frank

Aw: [RFC] pci: mediatek: add PCIe controller support for Filogic

[Frank Wunderlich]

a gentle ping...any comments?

regards Frank


> Gesendet: Freitag, 12. April 2024 um 16:10 Uhr
> Von: "Frank Wunderlich" 
> An: "Tom Rini" , "Lukasz Majewski" , "Sean 
> Anderson" , "Ryder Lee" , "Weijie 
> Gao" , "Chunfeng Yun" , 
> "GSS_MTK_Uboot_upstream" , "John 
> Crispin" 
> Cc: "Frank Wunderlich" , u-boot@lists.denx.de
> Betreff: [RFC] pci: mediatek: add PCIe controller support for Filogic
>
> From: John Crispin 
>
> This adds PCIe controller support for the MediaTek Filogic family..
>
> Signed-off-by: John Crispin 
> Signed-off-by: Frank Wunderlich 
> ---
> Note for mt7988: pcie2 needs a dedicated phy which has no driver
> in uboot yet, so this pcie port is not enabled in the board device-
> trees.
>
> Note for mt7981: i have no board and have no dts nodes yet for it,
> so only clock change first.
> ---
>  arch/arm/dts/mt7986.dtsi   |  46 +++
>  arch/arm/dts/mt7988-rfb.dts|  12 +
>  arch/arm/dts/mt7988-sd-rfb.dts |  12 +
>  arch/arm/dts/mt7988.dtsi   | 164 +++
>  drivers/clk/mediatek/clk-mt7986.c  |   5 +-
>  drivers/pci/Kconfig|   7 +
>  drivers/pci/Makefile   |   1 +
>  drivers/pci/pcie_mediatek_gen3.c   | 382 +
>  include/dt-bindings/clock/mt7981-clk.h |   3 +-
>  include/dt-bindings/clock/mt7986-clk.h |   3 +-
>  10 files changed, 631 insertions(+), 4 deletions(-)
>  create mode 100644 drivers/pci/pcie_mediatek_gen3.c
>
> diff --git a/arch/arm/dts/mt7986.dtsi b/arch/arm/dts/mt7986.dtsi
> index c9aeeaca2b11..9a9b0b64cc68 100644
> --- a/arch/arm/dts/mt7986.dtsi
> +++ b/arch/arm/dts/mt7986.dtsi
> @@ -375,5 +375,51 @@
>   #phy-cells = <1>;
>   status = "okay";
>   };
> +
> + pcie_port: pcie-phy@11c0 {
> + reg = <0x11c0 0x2>;
> + clocks = <_clk>;
> + clock-names = "ref";
> + #phy-cells = <1>;
> + status = "okay";
> + };
> + };
> +
> + pcie: pcie@1128 {
> + compatible = "mediatek,mt7986-pcie",
> +  "mediatek,mt8192-pcie";
> + device_type = "pci";
> + reg = <0x1128 0x4000>;
> + reg-names = "pcie-mac";
> + #address-cells = <3>;
> + #size-cells = <2>;
> +
> + clocks = <_ao CK_INFRA_IPCIE_PIPE_CK>,
> +  <_ao CK_INFRA_IPCIE_CK>,
> +  <_ao CK_INFRA_IPCIER_CK>,
> +  <_ao CK_INFRA_IPCIEB_CK>;
> + clock-names = "pl_250m", "tl_26m", "peri_26m", "top_133m";
> +
> + bus-range = <0x00 0xff>;
> + ranges = <0x8200 0 0x2000 0x2000 0 0x1000>;
> +
> + interrupts = ;
> + #interrupt-cells = <2>;
> + interrupt-map-mask = <0 0 0 7>;
> + interrupt-map = <0 0 0 1 _intc 0>, /* INTA */
> + <0 0 0 2 _intc 1>, /* INTB */
> + <0 0 0 3 _intc 2>, /* INTC */
> + <0 0 0 4 _intc 3>; /* INTD */
> +
> + phy-names = "pcie-phy";
> + phys = <_port PHY_TYPE_PCIE>;
> +
> + status = "okay";
> +
> + pcie_intc: legacy-interrupt-controller {
> + interrupt-controller;
> + #address-cells = <0>;
> + #interrupt-cells = <1>;
> + };
>   };
>  };
> diff --git a/arch/arm/dts/mt7988-rfb.dts b/arch/arm/dts/mt7988-rfb.dts
> index 2c1142843091..2f0d00b6950b 100644
> --- a/arch/arm/dts/mt7988-rfb.dts
> +++ b/arch/arm/dts/mt7988-rfb.dts
> @@ -180,3 +180,15 @@
>   non-removable;
>   status = "okay";
>  };
> +
> + {
> + status = "okay";
> +};
> +
> + {
> + status = "okay";
> +};
> +
> + {
> + status = "okay";
> +};
> diff --git a/arch/arm/dts/mt7988-sd-rfb.dts b/arch/arm/dts/mt7988-sd-rfb.dts
> index a3df37d252de..0a3eb5360d21 100644
> --- a/arch/arm/dts/mt7988-sd-rfb.dts
> +++ b/arch/arm/dts/mt7988-sd-rfb.dts
> @@ -132,3 +132,15 @@
>   vqmmc-supply = <_3p3v>;
>   status 

[DBWG] why can't i change role object?

[Frank Habicht]

Hi all,

I hope/trust that's not a question for "Support"...

I tried to update/modify a role object, and got this (sorry for the line 
breaks):



***Error:   Person/Role name cannot be changed automatically. Please create
another Person/Role object and modify any references to the old
object, then delete the old object



I can't see a reason why updates to role projects should not be allowed.
I even think that's the main purpose

So, am I missing something?


Thanks,
Frank



PS: AfriNIC staff:
I got "Internal software error" when i wanted to delete
SNHT5-AFRINIC
less than 1 minute after creating it

wanted to delete it with the line
delete: remove
added

according to
delete: 
under section "2.2.4 Deleting an object"
in
https://afrinic.net/press/197-database-afrinic-database-reference-manual-

deleting without the comment worked.

Can that section in that manual be reviewed?
If google sent me to an outdated document, can we remove it?


Thanks,
Frank

___
DBWG mailing list
DBWG@afrinic.net
https://lists.afrinic.net/mailman/listinfo/dbwg

[pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID

[Markus Frank]

Implement a systemd service that runs a C program that extracts AMD
SEV hardware information such as reduced-phys-bios and cbitpos from
CPUID at boot time, looks if SEV, SEV-ES & SEV-SNP are enabled, and
outputs these details as JSON to /run/qemu-server/hw-params.json.

This programm can also be used to read and save other hardware
information at boot time.

Signed-off-by: Markus Frank 
Co-authored-by: Thomas Lamprecht 
Tested-by: Filip Schauer 
---
changes v10:
* removed include of sys/types.h

 Makefile  |  1 +
 query-machine-capabilities/Makefile   | 21 ++
 .../query-machine-capabilities.c  | 70 +++
 .../query-machine-capabilities.service| 12 
 4 files changed, 104 insertions(+)
 create mode 100644 query-machine-capabilities/Makefile
 create mode 100644 query-machine-capabilities/query-machine-capabilities.c
 create mode 100644 
query-machine-capabilities/query-machine-capabilities.service

diff --git a/Makefile b/Makefile
index 133468d..ed67fe0 100644
--- a/Makefile
+++ b/Makefile
@@ -65,6 +65,7 @@ install: $(PKGSOURCES)
install -m 0644 -D bootsplash.jpg $(DESTDIR)/usr/share/$(PACKAGE)
$(MAKE) -C PVE install
$(MAKE) -C qmeventd install
+   $(MAKE) -C query-machine-capabilities install
$(MAKE) -C qemu-configs install
$(MAKE) -C vm-network-scripts install
install -m 0755 qm $(DESTDIR)$(SBINDIR)
diff --git a/query-machine-capabilities/Makefile 
b/query-machine-capabilities/Makefile
new file mode 100644
index 000..c5f6348
--- /dev/null
+++ b/query-machine-capabilities/Makefile
@@ -0,0 +1,21 @@
+DESTDIR=
+PREFIX=/usr
+SBINDIR=${PREFIX}/libexec/qemu-server
+SERVICEDIR=/lib/systemd/system
+
+CC ?= gcc
+CFLAGS += -O2 -fanalyzer -Werror -Wall -Wextra -Wpedantic -Wtype-limits 
-Wl,-z,relro -std=gnu11
+
+query-machine-capabilities: query-machine-capabilities.c
+   $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
+
+.PHONY: install
+install: query-machine-capabilities
+   install -d ${DESTDIR}/${SBINDIR}
+   install -d ${DESTDIR}${SERVICEDIR}
+   install -m 0644 query-machine-capabilities.service 
${DESTDIR}${SERVICEDIR}
+   install -m 0755 query-machine-capabilities ${DESTDIR}${SBINDIR}
+
+.PHONY: clean
+clean:
+   rm -f query-machine-capabilities
diff --git a/query-machine-capabilities/query-machine-capabilities.c 
b/query-machine-capabilities/query-machine-capabilities.c
new file mode 100644
index 000..4f18cde
--- /dev/null
+++ b/query-machine-capabilities/query-machine-capabilities.c
@@ -0,0 +1,70 @@
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+int main() {
+uint32_t eax, ebx, ecx, edx;
+
+// query Encrypted Memory Capabilities, see:
+// 
https://en.wikipedia.org/wiki/CPUID#EAX=801Fh:_Encrypted_Memory_Capabilities
+uint32_t query_function = 0x801F;
+asm volatile("cpuid"
+: "=a"(eax), "=b"(ebx), "=c"(ecx), "=d"(edx)
+: "0"(query_function)
+);
+
+bool sev_support = (eax & (1<<1)) != 0;
+bool sev_es_support = (eax & (1<<3)) != 0;
+bool sev_snp_support = (eax & (1<<4)) != 0;
+
+uint8_t cbitpos = ebx & 0x3f;
+uint8_t reduced_phys_bits = (ebx >> 6) & 0x3f;
+
+const char *path = "/run/qemu-server/";
+// Check that the directory exists and create it if it does not.
+struct stat statbuf;
+int stats = stat(path, );
+if (stats == 0 && S_ISDIR(statbuf.st_mode)) {
+   printf("Directory %s already exists.\n", path);
+} else if (errno == ENOENT) {
+   printf("%s does not exist. Creating directory.\n", path);
+   if (mkdir(path, 0755) != 0) {
+   printf("Error creating directory %s: %s\n", path, strerror(errno));
+   return 1;
+   }
+} else {
+   printf("Error checking path %s: %s\n", path, strerror(errno));
+   return 1;
+}
+
+FILE *file;
+const char *filename = "/run/qemu-server/host-hw-capabilities.json";
+file = fopen(filename, "w");
+if (file == NULL) {
+   perror("Error opening file");
+   return 1;
+}
+
+fprintf(file,
+   "{"
+   " \"amd-sev\": {"
+   " \"cbitpos\": %u,"
+   " \"reduced-phys-bits\": %u,"
+   " \"sev-support\": %s,"
+   " \"sev-support-es\": %s,"
+   " \"sev-support-snp\": %s"
+   " }"
+   " }\n",
+   cbitpos,
+   reduced_phys_bits,
+   sev_support ? "true" : "false",
+   sev_es_support ? "true" : "false",
+   sev_snp_support ? "true" : "false"
+);
+
+fclose(file);
+return 0;
+}
dif

[pve-devel] [PATCH qemu-server v10 2/4] config: add AMD SEV support

[Markus Frank]

This patch is for enabling AMD SEV (Secure Encrypted Virtualization)
support in QEMU.

VM-Config-Examples:
amd_sev: type=std,no-debug=1,no-key-sharing=1
amd_sev: es,no-debug=1,kernel-hashes=1

kernel-hashes, reduced-phys-bios & cbitpos correspond to the variables
with the same name in QEMU.

kernel-hashes=1 adds kernel-hashes to enable measured linux kernel
launch since it is per default off for backward compatibility.

reduced-phys-bios and cbitpos are system specific and are read out by
the query-machine-capabilities.service on boot and saved to the
/run/qemu-server/host-hw-capabilities.json file. This file is parsed
and than used by qemu-server to correctly start a AMD SEV VM.

type=std stands for standard sev to differentiate it from sev-es (es)
or sev-snp (snp) when support is upstream.

QEMU's sev-guest policy gets calculated with the parameters nodbg
& noks. These parameters correspond to policy-bits 0 & 1. If type is
'es' than policy-bit 2 gets set to 1 to activate SEV-ES. Policy bit 3
(nosend) is always set to 1, because migration features for sev are
not upstream yet and are attackable.

SEV-ES is highly experimental since it could not be tested.

see coherent doc patch

Signed-off-by: Markus Frank 
---
changes v10:
* also die if the BIOS is not set, since the default is SeaBIOS

 PVE/API2/Qemu.pm   | 11 +++
 PVE/QemuMigrate.pm |  4 +++
 PVE/QemuServer.pm  | 79 ++
 3 files changed, 94 insertions(+)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 2a349c8..c29809d 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4512,6 +4512,11 @@ __PACKAGE__->register_method({
push $local_resources->@*, "clipboard=vnc";
}
 
+   # do not allow live migration with AMD SEV enabled
+   if ($res->{running} && $vmconf->{amd_sev}) {
+   push $local_resources->@*, "amd_sev";
+   }
+
# if vm is not running, return target nodes where local storage/mapped 
devices are available
# for offline migration
if (!$res->{running}) {
@@ -5192,6 +5197,12 @@ __PACKAGE__->register_method({
die "unable to use snapshot name 'pending' (reserved name)\n"
if lc($snapname) eq 'pending';
 
+   my $conf = PVE::QemuConfig->load_config($vmid);
+   if ($param->{vmstate} && $conf->{amd_sev}) {
+   die "Snapshots that include memory are not supported while memory"
+   ." is encrypted by AMD SEV.\n"
+   }
+
my $realcmd = sub {
PVE::Cluster::log_msg('info', $authuser, "snapshot VM $vmid: 
$snapname");
PVE::QemuConfig->snapshot_create($vmid, $snapname, 
$param->{vmstate},
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 8d9b35a..340402a 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -260,6 +260,10 @@ sub prepare {
die "VMs with 'clipboard' set to 'vnc' are not live migratable!\n";
 }
 
+if ($running && $conf->{'amd_sev'}) {
+   die "cannot live-migrate VM when AMD SEV is enabled.\n";
+}
+
 my $vollist = PVE::QemuServer::get_vm_volumes($conf);
 
 my $storages = {};
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 82e7d6a..92960c5 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -177,6 +177,37 @@ my $agent_fmt = {
 },
 };
 
+my $sev_fmt = {
+type => {
+   description => "Enable standard SEV with type='std' or enable"
+   ." experimental SEV-ES with the 'es' option.",
+   type => 'string',
+   default_key => 1,
+   format_description => "sev-type",
+   enum => ['std', 'es'],
+   maxLength => 3,
+},
+'no-debug' => {
+   description => "Sets policy bit 0 to 1 to disallow debugging of guest",
+   type => 'boolean',
+   default => 0,
+   optional => 1,
+},
+'no-key-sharing' => {
+   description => "Sets policy bit 1 to 1 to disallow key sharing with 
other guests",
+   type => 'boolean',
+   default => 0,
+   optional => 1,
+},
+"kernel-hashes" => {
+   description => "Add kernel hashes to guest firmware for measured linux 
kernel launch",
+   type => 'boolean',
+   default => 0,
+   optional => 1,
+},
+};
+PVE::JSONSchema::register_format('pve-qemu-sev-fmt', $sev_fmt);
+
 my $vga_fmt = {
 type => {
description => "Select the VGA type.",
@@ -358,6 +389,12 @@ my $confdesc = {
description => "Memory properties.",
format => $PVE::QemuServer::Memory::memory_fmt
 },
+amd_sev => {
+   description => "Secure Encrypted Virtualization (SEV) features by AMD 
CPUs",
+   optional => 1,
+   

[pve-devel] [PATCH docs v10 3/4] add AMD SEV documentation

[Markus Frank]

add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV
enable" patch.

Signed-off-by: Markus Frank 
---
changes v10:
* none

 qm.adoc | 103 
 1 file changed, 103 insertions(+)

diff --git a/qm.adoc b/qm.adoc
index 42c26db..2001bd4 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -715,6 +715,109 @@ systems.
 When allocating RAM to your VMs, a good rule of thumb is always to leave 1GB
 of RAM available to the host.
 
+[[qm_memory_encryption]]
+Memory Encryption
+~
+
+[[qm_memory_encryption_sev]]
+AMD SEV
+^^^
+
+SEV (Secure Encrypted Virtualization) enables memory encryption per VM using
+AES-128 encryption and the AMD Secure Processor.
+
+SEV-ES (Secure Encrypted Virtualization-Encrypted State) in addition encrypts
+all CPU register contents when a VM stops running, to prevent leakage of
+information to the hypervisor. This feature is very experimental.
+
+*Host Requirements:*
+
+* AMD EPYC CPU
+* SEV-ES is only supported on AMD EPYC 7xx2 and newer
+* configure AMD memory encryption in the BIOS settings of the host machine
+* add "kvm_amd.sev=1" to kernel parameters if not enabled by default
+* add "mem_encrypt=on" to kernel parameters if you want to encrypt memory on 
the
+host (SME) see 
https://www.kernel.org/doc/Documentation/x86/amd-memory-encryption.txt
+* maybe increase SWIOTLB see https://github.com/AMDESE/AMDSEV#faq-4
+
+To check if SEV is enabled on the host search for `sev` in dmesg and print out
+the SEV kernel parameter of kvm_amd:
+
+
+# dmesg | grep -i sev
+[...] ccp :45:00.1: sev enabled
+[...] ccp :45:00.1: SEV API: 
+[...] SEV supported:  ASIDs
+[...] SEV-ES supported:  ASIDs
+# cat /sys/module/kvm_amd/parameters/sev
+Y
+
+
+*Guest Requirements:*
+
+* edk2-OVMF
+* advisable to use Q35
+* The guest operating system must contain SEV-support.
+
+*Limitations:*
+
+* Because the memory is encrypted the memory usage on host is always wrong.
+* Operations that involve saving or restoring memory like snapshots
+& live migration do not work yet or are attackable.
+https://github.com/PSPReverse/amd-sev-migration-attack
+* PCI passthrough is not supported.
+* SEV-ES is very experimental.
+* QEMU & AMD-SEV documentation is very limited.
+
+Example Configuration:
+
+
+# qm set  -amd_sev type=std,no-debug=1,no-key-sharing=1,kernel-hashes=1
+
+
+The *type* defines the encryption technology ("type=" is not necessary).
+Available options are std & es.
+
+The QEMU *policy* parameter gets calculated with the *no-debug* and
+*no-key-sharing* parameters. These parameters correspond to policy-bit 0 and 1.
+If *type* is *es* the policy-bit 2 is set to 1 so that SEV-ES is enabled.
+Policy-bit 3 (nosend) is always set to 1 to prevent migration-attacks. For more
+information on how to calculate the policy see:
+https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf[AMD
 SEV API Specification Chapter 3]
+
+The *kernel-hashes* is per default off for backward compatibility with older
+OVMF images and guests that do not measure the kernel/initrd.
+See https://lists.gnu.org/archive/html/qemu-devel/2021-11/msg02598.html
+
+*Check if SEV is working on the guest*
+
+Method 1 - dmesg:
+
+Output should look like this.
+
+
+# dmesg | grep -i sev
+AMD Memory Encryption Features active: SEV
+
+
+Method 2 - MSR 0xc0010131 (MSR_AMD64_SEV):
+
+Output should be 1.
+
+
+# apt install msr-tools
+# modprobe msr
+# rdmsr -a 0xc0010131
+1
+
+
+Links:
+
+* https://developer.amd.com/sev/
+* https://github.com/AMDESE/AMDSEV
+* https://www.qemu.org/docs/master/system/i386/amd-memory-encryption.html
+* https://www.amd.com/system/files/TechDocs/55766_SEV-KM_API_Specification.pdf
+* https://documentation.suse.com/sles/15-SP1/html/SLES-amd-sev/index.html
 
 [[qm_network_device]]
 Network Device
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server/docs/manager v10 0/4] AMD SEV

[Markus Frank]

Patch series to enable AMD Secure Encrypted Virtualization (SEV)

changes v10:
* removed include of sys/types.h in C Program
* also die if the BIOS is not set, since the default is SeaBIOS
* added pve-manager patch

apply/compile order:
1. qemu-server: add C program to get hardware capabilities from CPUID
2. qemu-server: config: add AMD SEV support
3. pve-docs: add AMD SEV documentation
4. pve-manager: ui: add AMD SEV configuration to Options


qemu-server:

Markus Frank (2):
  add C program to get hardware capabilities from CPUID
  config: add AMD SEV support

 Makefile  |  1 +
 PVE/API2/Qemu.pm  | 11 +++
 PVE/QemuMigrate.pm|  4 +
 PVE/QemuServer.pm | 79 +++
 query-machine-capabilities/Makefile   | 21 +
 .../query-machine-capabilities.c  | 70 
 .../query-machine-capabilities.service| 12 +++
 7 files changed, 198 insertions(+)
 create mode 100644 query-machine-capabilities/Makefile
 create mode 100644 query-machine-capabilities/query-machine-capabilities.c
 create mode 100644 
query-machine-capabilities/query-machine-capabilities.service


docs:

Markus Frank (1):
  add AMD SEV documentation

 qm.adoc | 103 
 1 file changed, 103 insertions(+)


manager:

Markus Frank (1):
  ui: add AMD SEV configuration to Options

 www/manager6/Makefile|  1 +
 www/manager6/qemu/Options.js | 11 
 www/manager6/qemu/SevEdit.js | 98 
 3 files changed, 110 insertions(+)
 create mode 100644 www/manager6/qemu/SevEdit.js

-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v10 4/4] ui: add AMD SEV configuration to Options

[Markus Frank]

By adding a new input panel with an AMD SEV technology selection combo
box and checkboxes for the optional parameters in an advanced section,
the user can configure the amd_sev option via the WebUI's Options tab.

Signed-off-by: Markus Frank 
---
changes v10:
* this patch is new to v10

 www/manager6/Makefile|  1 +
 www/manager6/qemu/Options.js | 11 
 www/manager6/qemu/SevEdit.js | 98 
 3 files changed, 110 insertions(+)
 create mode 100644 www/manager6/qemu/SevEdit.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 2c3a822b..801683a3 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -264,6 +264,7 @@ JSSRC=  
\
qemu/SSHKey.js  \
qemu/ScsiHwEdit.js  \
qemu/SerialEdit.js  \
+   qemu/SevEdit.js \
qemu/Smbios1Edit.js \
qemu/SystemEdit.js  \
qemu/USBEdit.js \
diff --git a/www/manager6/qemu/Options.js b/www/manager6/qemu/Options.js
index 7b112400..6907699c 100644
--- a/www/manager6/qemu/Options.js
+++ b/www/manager6/qemu/Options.js
@@ -338,6 +338,17 @@ Ext.define('PVE.qemu.Options', {
},
} : undefined,
},
+   amd_sev: {
+   header: gettext('AMD SEV'),
+   editor: caps.vms['VM.Config.HWType'] ? 'PVE.qemu.SevEdit' : 
undefined,
+   defaultValue: Proxmox.Utils.defaultText + ' (' + 
Proxmox.Utils.disabledText + ')',
+   renderer: function(value, metaData, record, ri, ci, store, 
pending) {
+   let amd_sev = PVE.Parser.parsePropertyString(value, "type");
+   if (amd_sev.type === 'std') return 'AMD SEV (' + value + 
')';
+   if (amd_sev.type === 'es') return 'AMD SEV-ES (' + value + 
')';
+   return value;
+   },
+   },
hookscript: {
header: gettext('Hookscript'),
},
diff --git a/www/manager6/qemu/SevEdit.js b/www/manager6/qemu/SevEdit.js
new file mode 100644
index ..f0187cde
--- /dev/null
+++ b/www/manager6/qemu/SevEdit.js
@@ -0,0 +1,98 @@
+Ext.define('PVE.qemu.SevInputPanel', {
+extend: 'Proxmox.panel.InputPanel',
+xtype: 'pveSevInputPanel',
+onlineHelp: 'qm_memory_encryption',
+
+viewModel: {
+   data: {
+   type: '__default__',
+   },
+   formulas: {
+   sevEnabled: get => get('type') === 'std' || get('type') === 'es',
+   },
+},
+
+onGetValues: function(values) {
+   if (values.delete === 'type') {
+   values.delete = 'amd_sev';
+   return values;
+   }
+   let ret = {};
+   ret.amd_sev = PVE.Parser.printPropertyString(values, 'type');
+   return ret;
+},
+
+items: {
+   xtype: 'proxmoxKVComboBox',
+   fieldLabel: gettext('AMD Secure Encrypted Virtualization (SEV)'),
+   name: 'type',
+   value: '__default__',
+   comboItems: [
+   ['__default__', Proxmox.Utils.defaultText + ' (' + 
Proxmox.Utils.disabledText + ')'],
+   ['std', 'AMD SEV'],
+   ['es', 'AMD SEV-ES (highly experimental)'],
+   ],
+   bind: {
+   value: '{type}',
+   },
+},
+
+advancedItems: [
+   {
+   xtype: 'proxmoxcheckbox',
+   fieldLabel: gettext('no-debug'),
+   name: 'no-debug',
+   deleteDefaultValue: false,
+   bind: {
+   hidden: '{!sevEnabled}',
+   disabled: '{!sevEnabled}',
+   },
+   },
+   {
+   xtype: 'proxmoxcheckbox',
+   fieldLabel: gettext('no-key-sharing'),
+   name: 'no-key-sharing',
+   deleteDefaultValue: false,
+   bind: {
+   hidden: '{!sevEnabled}',
+   disabled: '{!sevEnabled}',
+   },
+   },
+   {
+   xtype: 'proxmoxcheckbox',
+   fieldLabel: gettext('kernel-hashes'),
+   name: 'kernel-hashes',
+   deleteDefaultValue: false,
+   bind: {
+   hidden: '{!sevEnabled}',
+   disabled: '{!sevEnabled}',
+   },
+   },
+],
+});
+
+Ext.define('PVE.qemu.SevEdit', {
+extend: 'Proxmox.window.Edit',
+
+subject: gettext('SEV'),
+
+items: {
+   xtype: 'pveSevInputPanel',
+},
+
+width: 400,
+
+initComponent: function() {
+   let me = this;
+
+   me.callParent();
+
+   me.load({
+   success: function(response) {
+   let conf = response.result.data;
+   let amd_sev = conf.amd_sev || '__default__';
+   me.setValues(PVE.Parser.parsePropertyString(amd_sev, 'type'));
+   },
+   });
+},
+}

Re: [PATCH v2 00/15] riscv: QEMU RISC-V IOMMU Support

[Frank Chang]

Hi Daniel,

Thanks for the upstream work.
Sorry that it took a while for me to review the patchset.

Please let me know if you need any help from us to update the IOMMU model.
We would like to see it merged for QEMU 9.1.0.

Regards,
Frank Chang

Daniel Henrique Barboza  於 2024年3月8日 週五 上午12:04寫道：
>
> Hi,
>
> This is the second version of the work Tomasz sent in July 2023 [1].
> I'll be helping Tomasz upstreaming it.
>
> The core emulation code is left unchanged but a few tweaks were made in
> v2:
>
> - The most notable difference in this version is that the code was split
>   in smaller chunks. Patch 03 is still a 1700 lines patch, which is an
>   improvement from the 3800 lines patch from v1, but we can only go so
>   far when splitting the core components of the emulation. The reality
>   is that the IOMMU emulation is a rather complex piece of software and
>   there's not much we can do to alleviate it;
>
> - I'm not contributing the HPM support that was present in v1. It shaved
>   off 600 lines of code from the series, which is already large enough
>   as is. We'll introduce HPM in later versions or as a follow-up;
>
> - The riscv-iommu-header.h header was also trimmed. I shaved it of 300
>   or so from it, all of them from definitions that the emulation isn't
>   using it. The header will be eventually be imported from the Linux
>   driver (not upstream yet), so for now we can live with a trimmed
>   header for the emulation usage alone;
>
> - I added libqos tests for the riscv-iommu-pci device. The idea of these
>   tests is to give us more confidence in the emulation code;
>
> - 'edu' device support. The support was retrieved from Tomasz EDU branch
>   [2]. This device can then be used to test PCI passthrough to exercise
>   the IOMMU.
>
>
> Patches based on alistair/riscv-to-apply.next.
>
> v1 link: 
> https://lore.kernel.org/qemu-riscv/cover.1689819031.git.tjezn...@rivosinc.com/
>
> [1] 
> https://lore.kernel.org/qemu-riscv/cover.1689819031.git.tjezn...@rivosinc.com/
> [2] https://github.com/tjeznach/qemu.git, branch 'riscv_iommu_edu_impl'
>
> Andrew Jones (1):
>   hw/riscv/riscv-iommu: Add another irq for mrif notifications
>
> Daniel Henrique Barboza (2):
>   test/qtest: add riscv-iommu-pci tests
>   qtest/riscv-iommu-test: add init queues test
>
> Tomasz Jeznach (12):
>   exec/memtxattr: add process identifier to the transaction attributes
>   hw/riscv: add riscv-iommu-bits.h
>   hw/riscv: add RISC-V IOMMU base emulation
>   hw/riscv: add riscv-iommu-pci device
>   hw/riscv: add riscv-iommu-sys platform device
>   hw/riscv/virt.c: support for RISC-V IOMMU PCIDevice hotplug
>   hw/riscv/riscv-iommu: add Address Translation Cache (IOATC)
>   hw/riscv/riscv-iommu: add s-stage and g-stage support
>   hw/riscv/riscv-iommu: add ATS support
>   hw/riscv/riscv-iommu: add DBG support
>   hw/misc: EDU: added PASID support
>   hw/misc: EDU: add ATS/PRI capability
>
>  hw/misc/edu.c|  297 -
>  hw/riscv/Kconfig |4 +
>  hw/riscv/meson.build |1 +
>  hw/riscv/riscv-iommu-bits.h  |  407 ++
>  hw/riscv/riscv-iommu-pci.c   |  173 +++
>  hw/riscv/riscv-iommu-sys.c   |   93 ++
>  hw/riscv/riscv-iommu.c   | 2085 ++
>  hw/riscv/riscv-iommu.h   |  146 +++
>  hw/riscv/trace-events|   15 +
>  hw/riscv/trace.h |2 +
>  hw/riscv/virt.c  |   33 +-
>  include/exec/memattrs.h  |5 +
>  include/hw/riscv/iommu.h |   40 +
>  meson.build  |1 +
>  tests/qtest/libqos/meson.build   |4 +
>  tests/qtest/libqos/riscv-iommu.c |   79 ++
>  tests/qtest/libqos/riscv-iommu.h |   96 ++
>  tests/qtest/meson.build  |1 +
>  tests/qtest/riscv-iommu-test.c   |  234 
>  19 files changed, 3704 insertions(+), 12 deletions(-)
>  create mode 100644 hw/riscv/riscv-iommu-bits.h
>  create mode 100644 hw/riscv/riscv-iommu-pci.c
>  create mode 100644 hw/riscv/riscv-iommu-sys.c
>  create mode 100644 hw/riscv/riscv-iommu.c
>  create mode 100644 hw/riscv/riscv-iommu.h
>  create mode 100644 hw/riscv/trace-events
>  create mode 100644 hw/riscv/trace.h
>  create mode 100644 include/hw/riscv/iommu.h
>  create mode 100644 tests/qtest/libqos/riscv-iommu.c
>  create mode 100644 tests/qtest/libqos/riscv-iommu.h
>  create mode 100644 tests/qtest/riscv-iommu-test.c
>
> --
> 2.43.2
>
>

Re: [PATCH v2 02/15] hw/riscv: add riscv-iommu-bits.h

[Frank Chang]

Reviewed-by: Frank Chang 

Daniel Henrique Barboza  於 2024年3月8日 週五 上午12:07寫道：
>
> From: Tomasz Jeznach 
>
> This header will be used by the RISC-V IOMMU emulation to be added
> in the next patch. Due to its size it's being sent in separate for
> an easier review.
>
> One thing to notice is that this header can be replaced by the future
> Linux RISC-V IOMMU driver header, which would become a linux-header we
> would import instead of keeping our own. The Linux implementation isn't
> upstream yet so for now we'll have to manage riscv-iommu-bits.h.
>
> Signed-off-by: Tomasz Jeznach 
> Signed-off-by: Daniel Henrique Barboza 
> ---
>  hw/riscv/riscv-iommu-bits.h | 335 
>  1 file changed, 335 insertions(+)
>  create mode 100644 hw/riscv/riscv-iommu-bits.h
>
> diff --git a/hw/riscv/riscv-iommu-bits.h b/hw/riscv/riscv-iommu-bits.h
> new file mode 100644
> index 00..8e80b1e52a
> --- /dev/null
> +++ b/hw/riscv/riscv-iommu-bits.h
> @@ -0,0 +1,335 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright © 2022-2023 Rivos Inc.
> + * Copyright © 2023 FORTH-ICS/CARV
> + * Copyright © 2023 RISC-V IOMMU Task Group
> + *
> + * RISC-V Ziommu - Register Layout and Data Structures.
> + *
> + * Based on the IOMMU spec version 1.0, 3/2023
> + * https://github.com/riscv-non-isa/riscv-iommu
> + */
> +
> +#ifndef HW_RISCV_IOMMU_BITS_H
> +#define HW_RISCV_IOMMU_BITS_H
> +
> +#include "qemu/osdep.h"
> +
> +#define RISCV_IOMMU_SPEC_DOT_VER 0x010
> +
> +#ifndef GENMASK_ULL
> +#define GENMASK_ULL(h, l) (((~0ULL) >> (63 - (h) + (l))) << (l))
> +#endif
> +
> +/*
> + * struct riscv_iommu_fq_record - Fault/Event Queue Record
> + * See section 3.2 for more info.
> + */
> +struct riscv_iommu_fq_record {
> +uint64_t hdr;
> +uint64_t _reserved;
> +uint64_t iotval;
> +uint64_t iotval2;
> +};
> +/* Header fields */
> +#define RISCV_IOMMU_FQ_HDR_CAUSEGENMASK_ULL(11, 0)
> +#define RISCV_IOMMU_FQ_HDR_PID  GENMASK_ULL(31, 12)
> +#define RISCV_IOMMU_FQ_HDR_PV   BIT_ULL(32)
> +#define RISCV_IOMMU_FQ_HDR_TTYPEGENMASK_ULL(39, 34)
> +#define RISCV_IOMMU_FQ_HDR_DID  GENMASK_ULL(63, 40)
> +
> +/*
> + * struct riscv_iommu_pq_record - PCIe Page Request record
> + * For more infos on the PCIe Page Request queue see chapter 3.3.
> + */
> +struct riscv_iommu_pq_record {
> +  uint64_t hdr;
> +  uint64_t payload;
> +};
> +/* Header fields */
> +#define RISCV_IOMMU_PREQ_HDR_PIDGENMASK_ULL(31, 12)
> +#define RISCV_IOMMU_PREQ_HDR_PV BIT_ULL(32)
> +#define RISCV_IOMMU_PREQ_HDR_PRIV   BIT_ULL(33)
> +#define RISCV_IOMMU_PREQ_HDR_EXEC   BIT_ULL(34)
> +#define RISCV_IOMMU_PREQ_HDR_DIDGENMASK_ULL(63, 40)
> +/* Payload fields */
> +#define RISCV_IOMMU_PREQ_PAYLOAD_M  GENMASK_ULL(2, 0)
> +
> +/* Common field positions */
> +#define RISCV_IOMMU_PPN_FIELD   GENMASK_ULL(53, 10)
> +#define RISCV_IOMMU_QUEUE_LOGSZ_FIELD   GENMASK_ULL(4, 0)
> +#define RISCV_IOMMU_QUEUE_INDEX_FIELD   GENMASK_ULL(31, 0)
> +#define RISCV_IOMMU_QUEUE_ENABLEBIT(0)
> +#define RISCV_IOMMU_QUEUE_INTR_ENABLE   BIT(1)
> +#define RISCV_IOMMU_QUEUE_MEM_FAULT BIT(8)
> +#define RISCV_IOMMU_QUEUE_OVERFLOW  BIT(9)
> +#define RISCV_IOMMU_QUEUE_ACTIVEBIT(16)
> +#define RISCV_IOMMU_QUEUE_BUSY  BIT(17)
> +#define RISCV_IOMMU_ATP_PPN_FIELD   GENMASK_ULL(43, 0)
> +#define RISCV_IOMMU_ATP_MODE_FIELD  GENMASK_ULL(63, 60)
> +
> +/* 5.3 IOMMU Capabilities (64bits) */
> +#define RISCV_IOMMU_REG_CAP 0x
> +#define RISCV_IOMMU_CAP_VERSION GENMASK_ULL(7, 0)
> +#define RISCV_IOMMU_CAP_MSI_FLATBIT_ULL(22)
> +#define RISCV_IOMMU_CAP_MSI_MRIFBIT_ULL(23)
> +#define RISCV_IOMMU_CAP_IGS GENMASK_ULL(29, 28)
> +#define RISCV_IOMMU_CAP_PAS GENMASK_ULL(37, 32)
> +#define RISCV_IOMMU_CAP_PD8 BIT_ULL(38)
> +
> +/* 5.4 Features control register (32bits) */
> +#define RISCV_IOMMU_REG_FCTL0x0008
> +
> +/* 5.5 Device-directory-table pointer (64bits) */
> +#define RISCV_IOMMU_REG_DDTP0x0010
> +#define RISCV_IOMMU_DDTP_MODE   GENMASK_ULL(3, 0)
> +#define RISCV_IOMMU_DDTP_BUSY   BIT_ULL(4)
> +#define RISCV_IOMMU_DDTP_PPNRISCV_IOMMU_PPN_FIELD
> +
> +enum riscv_iommu_ddtp_modes {
> +RISCV_IOMMU_DDTP_MODE_OFF = 0,
> +RISCV_IOMMU_DDTP_MODE_BARE = 1,
> +RISCV_IOMMU_DDTP_MODE_1LVL = 2,
> +RISCV_IOMMU_DDTP_MODE_2LVL = 3,
> +RISCV_IOMMU_DDTP_MODE_3LVL = 4,
> +RISCV_IOMMU_DDTP_MODE_MAX = 4
> +};
> +
> +/* 5.

Re: [PATCH v2 11/15] hw/riscv/riscv-iommu: add DBG support

[Frank Chang]

Hi Daniel,

Daniel Henrique Barboza  於 2024年5月6日 週一 下午9:06寫道：
>
> Hi Frank,
>
> On 5/6/24 01:09, Frank Chang wrote:
> > Hi Daniel,
> >
> > Daniel Henrique Barboza  於 2024年3月8日 週五 
> > 上午12:05寫道：
> >>
> >> From: Tomasz Jeznach 
> >>
> >> DBG support adds three additional registers: tr_req_iova, tr_req_ctl and
> >> tr_response.
> >>
> >> The DBG cap is always enabled. No on/off toggle is provided for it.
> >>
> >> Signed-off-by: Tomasz Jeznach 
> >> Signed-off-by: Daniel Henrique Barboza 
> >> ---
> >>   hw/riscv/riscv-iommu-bits.h | 20 +
> >>   hw/riscv/riscv-iommu.c  | 57 -
> >>   2 files changed, 76 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/hw/riscv/riscv-iommu-bits.h b/hw/riscv/riscv-iommu-bits.h
> >> index 0994f5ce48..b3f92411bb 100644
> >> --- a/hw/riscv/riscv-iommu-bits.h
> >> +++ b/hw/riscv/riscv-iommu-bits.h
> >> @@ -83,6 +83,7 @@ struct riscv_iommu_pq_record {
> >>   #define RISCV_IOMMU_CAP_MSI_MRIFBIT_ULL(23)
> >>   #define RISCV_IOMMU_CAP_ATS BIT_ULL(25)
> >>   #define RISCV_IOMMU_CAP_IGS GENMASK_ULL(29, 28)
> >> +#define RISCV_IOMMU_CAP_DBG BIT_ULL(31)
> >>   #define RISCV_IOMMU_CAP_PAS GENMASK_ULL(37, 32)
> >>   #define RISCV_IOMMU_CAP_PD8 BIT_ULL(38)
> >>
> >> @@ -177,6 +178,25 @@ enum {
> >>   RISCV_IOMMU_INTR_COUNT
> >>   };
> >>
> >> +#define RISCV_IOMMU_IPSR_CIPBIT(RISCV_IOMMU_INTR_CQ)
> >> +#define RISCV_IOMMU_IPSR_FIPBIT(RISCV_IOMMU_INTR_FQ)
> >> +#define RISCV_IOMMU_IPSR_PMIP   BIT(RISCV_IOMMU_INTR_PM)
> >> +#define RISCV_IOMMU_IPSR_PIPBIT(RISCV_IOMMU_INTR_PQ)
> >
> > These are not related to the DBG.
> >
> >> +
> >> +/* 5.24 Translation request IOVA (64bits) */
> >> +#define RISCV_IOMMU_REG_TR_REQ_IOVA 0x0258
> >> +
> >> +/* 5.25 Translation request control (64bits) */
> >> +#define RISCV_IOMMU_REG_TR_REQ_CTL  0x0260
> >> +#define RISCV_IOMMU_TR_REQ_CTL_GO_BUSY  BIT_ULL(0)
> >> +#define RISCV_IOMMU_TR_REQ_CTL_PID  GENMASK_ULL(31, 12)
> >> +#define RISCV_IOMMU_TR_REQ_CTL_DID  GENMASK_ULL(63, 40)
> >> +
> >> +/* 5.26 Translation request response (64bits) */
> >> +#define RISCV_IOMMU_REG_TR_RESPONSE 0x0268
> >> +#define RISCV_IOMMU_TR_RESPONSE_FAULT   BIT_ULL(0)
> >> +#define RISCV_IOMMU_TR_RESPONSE_PPN RISCV_IOMMU_PPN_FIELD
> >> +
> >>   /* 5.27 Interrupt cause to vector (64bits) */
> >>   #define RISCV_IOMMU_REG_IVEC0x02F8
> >>
> >> diff --git a/hw/riscv/riscv-iommu.c b/hw/riscv/riscv-iommu.c
> >> index 7af5929b10..1fa1286d07 100644
> >> --- a/hw/riscv/riscv-iommu.c
> >> +++ b/hw/riscv/riscv-iommu.c
> >> @@ -1457,6 +1457,46 @@ static void 
> >> riscv_iommu_process_pq_control(RISCVIOMMUState *s)
> >>   riscv_iommu_reg_mod32(s, RISCV_IOMMU_REG_PQCSR, ctrl_set, ctrl_clr);
> >>   }
> >>
> >> +static void riscv_iommu_process_dbg(RISCVIOMMUState *s)
> >> +{
> >> +uint64_t iova = riscv_iommu_reg_get64(s, RISCV_IOMMU_REG_TR_REQ_IOVA);
> >> +uint64_t ctrl = riscv_iommu_reg_get64(s, RISCV_IOMMU_REG_TR_REQ_CTL);
> >> +unsigned devid = get_field(ctrl, RISCV_IOMMU_TR_REQ_CTL_DID);
> >> +unsigned pid = get_field(ctrl, RISCV_IOMMU_TR_REQ_CTL_PID);
> >> +RISCVIOMMUContext *ctx;
> >> +void *ref;
> >> +
> >> +if (!(ctrl & RISCV_IOMMU_TR_REQ_CTL_GO_BUSY)) {
> >> +return;
> >> +}
> >> +
> >> +ctx = riscv_iommu_ctx(s, devid, pid, );
> >> +if (ctx == NULL) {
> >> +riscv_iommu_reg_set64(s, RISCV_IOMMU_REG_TR_RESPONSE,
> >> + RISCV_IOMMU_TR_RESPONSE_FAULT |
> >> + (RISCV_IOMMU_FQ_CAUSE_DMA_DISABLED << 
> >> 10));
> >> +} else {
> >> +IOMMUTLBEntry iotlb = {
> >> +.iova = iova,
> >> +.perm = IOMMU_NONE,
> >
> > .perm should honor tr_req_ctl.[Exe|Nw]
> >
> >> +.addr_mask = ~0,
> >> +.target_as = NULL,
> >> +};
> >> +int fault = riscv_iommu_translate(s, ctx, , false);
> >> +if (fault) {
> >> +iova = RIS

Re: [PATCH v2 03/15] hw/riscv: add RISC-V IOMMU base emulation

[Frank Chang]

Hi Daniel,

Daniel Henrique Barboza  於 2024年5月8日 週三 下午7:16寫道：
>
> Hi Frank,
>
> I'll reply with that I've done so far. Still missing some stuff:
>
> On 5/2/24 08:37, Frank Chang wrote:
> > Hi Daniel,
> >
> > Daniel Henrique Barboza  於 2024年3月8日 週五 
> > 上午12:04寫道：
> >>
> >> From: Tomasz Jeznach 
> >>
> >> The RISC-V IOMMU specification is now ratified as-per the RISC-V
> >> international process. The latest frozen specifcation can be found
> >> at:
> >>
> >> https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0/riscv-iommu.pdf
> >>
> >> Add the foundation of the device emulation for RISC-V IOMMU, which
> >> includes an IOMMU that has no capabilities but MSI interrupt support and
> >> fault queue interfaces. We'll add add more features incrementally in the
> >> next patches.
> >>
> >> Co-developed-by: Sebastien Boeuf 
> >> Signed-off-by: Sebastien Boeuf 
> >> Signed-off-by: Tomasz Jeznach 
> >> Signed-off-by: Daniel Henrique Barboza 
> >> ---
> >>   hw/riscv/Kconfig |4 +
> >>   hw/riscv/meson.build |1 +
> >>   hw/riscv/riscv-iommu.c   | 1492 ++
> >>   hw/riscv/riscv-iommu.h   |  141 
> >>   hw/riscv/trace-events|   11 +
> >>   hw/riscv/trace.h |2 +
> >>   include/hw/riscv/iommu.h |   36 +
> >>   meson.build  |1 +
> >>   8 files changed, 1688 insertions(+)
> >>   create mode 100644 hw/riscv/riscv-iommu.c
> >>   create mode 100644 hw/riscv/riscv-iommu.h
> >>   create mode 100644 hw/riscv/trace-events
> >>   create mode 100644 hw/riscv/trace.h
> >>   create mode 100644 include/hw/riscv/iommu.h
> >>
>
> (...)
>
> +{
> >> +const uint32_t ipsr =
> >> +riscv_iommu_reg_mod32(s, RISCV_IOMMU_REG_IPSR, (1 << vec), 0);
> >> +const uint32_t ivec = riscv_iommu_reg_get32(s, RISCV_IOMMU_REG_IVEC);
> >> +if (s->notify && !(ipsr & (1 << vec))) {
> >> +s->notify(s, (ivec >> (vec * 4)) & 0x0F);
> >> +}
> >
> > s->notify is assigned to riscv_iommu_pci_notify() only.
> > There's no way to assert the wire-signaled interrupt.
> >
> > We should also check fctl.WSI before asserting the interrupt.
> >
>
> This implementation does not support wire-signalled interrupts. It supports 
> only
> MSI, i.e. capabililities.IGS is always MSI (0). For this reason the code is 
> also
> not checking for fctl.WSI.
>
>
>
> >> +}
>   (...)
>
> >> +g_hash_table_unref(ctx_cache);
> >> +*ref = NULL;
> >> +
> >> +if (!(ctx->tc & RISCV_IOMMU_DC_TC_DTF)) {
> >
> > riscv_iommu_ctx_fetch() may return:
> > RISCV_IOMMU_FQ_CAUSE_DMA_DISABLED (256)
> > RISCV_IOMMU_FQ_CAUSE_DDT_LOAD_FAULT (257)
> > RISCV_IOMMU_FQ_CAUSE_DDT_INVALID (258)
> > RISCV_IOMMU_FQ_CAUSE_DDT_MISCONFIGURED (259)
> >
> > These faults are reported even when DTF is set to 1.
> > We should report these faults regardless of DTF setting.
>
>
> I created a "riscv_iommu_report_fault()" helper to centralize all the report 
> fault
> logic. This helper will check for DTF and, if set, we'll check the 'cause' to 
> see if
> we still want the fault to be reported or not. This helper is then used in 
> these 2
> instances where we're creating a fault by hand. It's also used extensively in
> riscv_iommu_msi_write() to handle all the cases you mentioned above where we
> weren't issuing faults.
>
>
> >
> >> +struct riscv_iommu_fq_record ev = { 0 };
> >> +ev.hdr = set_field(ev.hdr, RISCV_IOMMU_FQ_HDR_CAUSE, fault);
> >> +ev.hdr = set_field(ev.hdr, RISCV_IOMMU_FQ_HDR_TTYPE,
> >> +RISCV_IOMMU_FQ_TTYPE_UADDR_RD);
> >> +ev.hdr = set_field(ev.hdr, RISCV_IOMMU_FQ_HDR_DID, devid);
> >> +ev.hdr = set_field(ev.hdr, RISCV_IOMMU_FQ_HDR_PID, pasid);
> >> +ev.hdr = set_field(ev.hdr, RISCV_IOMMU_FQ_HDR_PV, !!pasid);
> >> +riscv_iommu_fault(s, );
> >> +}
> >> +
> >> +g_free(ctx);
> >> +return NULL;
> >> +}
> >> +
> >> +static void riscv_iommu_ctx_put(RISCVIOMMUState *s, void *ref)
> >> +{
> >> +if (ref) {
> >> +g_hash_table_unref((GHashTable *)ref);
> >> +}
> >> +}
> >> +
> >> +/* Find or allocate address spa

Re: [RFC PATCH 0/1] pci: allocate a PCI ID for RISC-V IOMMU

[Frank Chang]

Hi Daniel,

Daniel Henrique Barboza  於 2024年5月8日 週三 下午8:42寫道：
>
>
>
> On 5/7/24 12:44, Peter Maydell wrote:
> > On Fri, 3 May 2024 at 13:43, Daniel Henrique Barboza
> >  wrote:
> >>
> >> Hi,
> >>
> >> In this RFC I want to check with Gerd and others if it's ok to add a PCI
> >> id for the RISC-V IOMMU device. It's currently under review in [1]. The
> >> idea is to fold this patch into the RISC-V IOMMU series if we're all ok
> >> with this change.
> >
> > My question here would be "why is this risc-v specific?" (and more
> > generally "what is this for?" -- the cover letter and patch and
> > documentation page provide almost no information about what this
> > device is and why it needs to exist rather than using either
> > virtio-iommu or else a model of a real hardware IOMMU.)
>
> The RISC-V IOMMU device emulation under review ([1]) is a reference 
> implementation of
> the riscv-iommu spec [2]. AFAIK it is similar to what we already have with 
> aarch64 'smmuv3'
> 'virt' bus, i.e. an impl of ARM's SMMUv3 that isn't tied to a specific vendor.
>
> The difference here is that the riscv-iommu spec, ratified by RISC-V 
> International (RVI),
> predicts that the device could be implemented as a PCIe device. But RVI 
> didn't bother
> assigning a PCI ID for their reference IOMMU. The existing implementation in 
> [1] is using
> a Rivos PCI ID that we're treating as a placeholder only. We need an ID that 
> reflects that
> this is a device that adheres to the riscv-iommu spec, not to an IOMMU of any 
> particular
> vendor.
>
> Since RVI doesn't provide a PCI ID for it we went to Red Hat, and they were 
> kind enough
> to give us a PCI ID for the RISC-V IOMMU reference device.

That's great. Thanks to Red Hat.
I'm wondering do we have the plan to document the new PCI ID to the IOMMU spec
or somewhere else that's publicly accessible?

Regards,
Frank Chang

>
> I'll do a proper job this time and add all this context in the commit msg. 
> Including a
> proper shout-out to Gerd and Red Hat.
>
>
>
> Thanks,
>
>
> Daniel
>
>
> [1] 
> https://lore.kernel.org/qemu-riscv/20240307160319.675044-1-dbarb...@ventanamicro.com/
> [2] https://github.com/riscv-non-isa/riscv-iommu/releases/tag/v1.0.0
>
> >
> > thanks
> > -- PMM
>

Re: [PATCH v2 09/15] hw/riscv/riscv-iommu: add s-stage and g-stage support

[Frank Chang]

   riscv_iommu_msi_check(s, ctx, base)) {
> +/* Trap MSI writes and return GPA address. */
> +iotlb->target_as = >trap_as;
> +iotlb->addr_mask = ~TARGET_PAGE_MASK;
> +return 0;
> +}
> +
> +/* Continue with G-Stage translation? */
> +if (!pass && en_g) {
> +pass = G_STAGE;
> +addr = base;
> +base = gatp;
> +sc[pass].step = 0;
> +continue;
> +}
> +
> +return 0;
> +}
> +
> +if (sc[pass].step == sc[pass].levels) {
> +break; /* Can't find leaf PTE */
> +}
> +
> +/* Continue with G-Stage translation? */
> +if (!pass && en_g) {
> +pass = G_STAGE;
> +addr = base;
> +base = gatp;
> +sc[pass].step = 0;
> +}

Will this if condition ever be executed?

For S-stage -> G-stage (i.e. Nested translation),
G-stage translation should be continued by
the S-stage Leaf PTE's if condition above?

> +} while (1);
> +
> +return (iotlb->perm & IOMMU_WO) ?
> +(pass ? RISCV_IOMMU_FQ_CAUSE_WR_FAULT_VS :
> +RISCV_IOMMU_FQ_CAUSE_WR_FAULT_S) :
> +(pass ? RISCV_IOMMU_FQ_CAUSE_RD_FAULT_VS :
> +RISCV_IOMMU_FQ_CAUSE_RD_FAULT_S);
>  }
>
>  /* Redirect MSI write for given GPA. */
> @@ -351,6 +572,10 @@ static int riscv_iommu_ctx_fetch(RISCVIOMMUState *s, 
> RISCVIOMMUContext *ctx)
>
>  case RISCV_IOMMU_DDTP_MODE_BARE:
>  /* mock up pass-through translation context */
> +ctx->gatp = set_field(0, RISCV_IOMMU_ATP_MODE_FIELD,
> +RISCV_IOMMU_DC_IOHGATP_MODE_BARE);
> +ctx->satp = set_field(0, RISCV_IOMMU_ATP_MODE_FIELD,
> +RISCV_IOMMU_DC_FSC_MODE_BARE);
>  ctx->tc = RISCV_IOMMU_DC_TC_V;
>  ctx->ta = 0;
>  ctx->msiptp = 0;
> @@ -424,6 +649,8 @@ static int riscv_iommu_ctx_fetch(RISCVIOMMUState *s, 
> RISCVIOMMUContext *ctx)
>
>  /* Set translation context. */
>  ctx->tc = le64_to_cpu(dc.tc);
> +ctx->gatp = le64_to_cpu(dc.iohgatp);
> +ctx->satp = le64_to_cpu(dc.fsc);
>  ctx->ta = le64_to_cpu(dc.ta);
>  ctx->msiptp = le64_to_cpu(dc.msiptp);
>  ctx->msi_addr_mask = le64_to_cpu(dc.msi_addr_mask);
> @@ -433,14 +660,38 @@ static int riscv_iommu_ctx_fetch(RISCVIOMMUState *s, 
> RISCVIOMMUContext *ctx)
>  return RISCV_IOMMU_FQ_CAUSE_DDT_INVALID;
>  }
>
> +/* FSC field checks */
> +mode = get_field(ctx->satp, RISCV_IOMMU_DC_FSC_MODE);
> +addr = PPN_PHYS(get_field(ctx->satp, RISCV_IOMMU_DC_FSC_PPN));
> +
> +if (mode == RISCV_IOMMU_DC_FSC_MODE_BARE) {
> +/* No S-Stage translation, done. */
> +return 0;
> +}
> +
>  if (!(ctx->tc & RISCV_IOMMU_DC_TC_PDTV)) {
>  if (ctx->pasid != RISCV_IOMMU_NOPASID) {
>  /* PASID is disabled */
>  return RISCV_IOMMU_FQ_CAUSE_TTYPE_BLOCKED;
>  }
> +if (mode > RISCV_IOMMU_DC_FSC_IOSATP_MODE_SV57) {
> +/* Invalid translation mode */
> +return RISCV_IOMMU_FQ_CAUSE_DDT_INVALID;
> +}
>  return 0;
>  }
>
> +if (ctx->pasid == RISCV_IOMMU_NOPASID) {
> +if (!(ctx->tc & RISCV_IOMMU_DC_TC_DPE)) {
> +/* No default PASID enabled, set BARE mode */
> +ctx->satp = 0ULL;
> +return 0;
> +} else {
> +/* Use default PASID #0 */
> +ctx->pasid = 0;

How do we differentiate between the default PASID: 0
and RISCV_IOMMU_NOPASID?

Regards,
Frank Chang

> +}
> +}
> +
>  /* FSC.TC.PDTV enabled */
>  if (mode > RISCV_IOMMU_DC_FSC_PDTP_MODE_PD20) {
>  /* Invalid PDTP.MODE */
> @@ -474,6 +725,7 @@ static int riscv_iommu_ctx_fetch(RISCVIOMMUState *s, 
> RISCVIOMMUContext *ctx)
>
>  /* Use FSC and TA from process directory entry. */
>  ctx->ta = le64_to_cpu(dc.ta);
> +ctx->satp = le64_to_cpu(dc.fsc);
>
>  return 0;
>  }
> @@ -710,6 +962,7 @@ static RISCVIOMMUEntry 
> *riscv_iommu_iot_lookup(RISCVIOMMUContext *ctx,
>  GHashTable *iot_cache, hwaddr iova)
>  {
>  RISCVIOMMUEntry key = {
> +.gscid = get_field(ctx->gatp, RISCV_IOMMU_DC_IOHGATP_GSCID),
>  .pscid = get_field(ctx->ta, RISCV_IOMMU_DC_TA_PSCID),
>  .iova  = PPN_DOWN(iova),
>  };
> @@ -779,7 +1032,7 @@ static int riscv_iommu_translate(RISCV

FLINK-33759

[Frank Yin]

Hi,

I opened a PR on GitHub to address FLINK-33759 that causes issues when
writing Parquet files with complex data schema.
https://github.com/apache/flink/pull/24029

Can anyone help review this PR?

Thanks,
Frank

Re: [PATCH] Fix 'make coverage' when used with lcov version 2.0+

[Frank Ch. Eigler]

Hi -

> Well, even if it happened unintentionally, it was ignored just the same
> as if it happened on purpose.  I hope the technical reasons for that
> are now fixed, and won't cause any problems next time.

You have been in the open source community long enough to know that
emails sometimes just get lost.  It might happen again, and one one
should not take too much offence.

> Please note that elfutils is a community project, and the code in
> question was contributed a few years ago by myself, see commit
> 2a16a0fc7e353f8fcfc27a57710e008840297847.

Yes, however we both know that this does not mean that a maintainer is
required to affirmatively consult you (or anyone) on changes years
after its contribution.

> Frank, please reconsider your approach.  Your comments are [...]

I disagree.


- FChE

Re: New installation of Cygwin64: xinit.sh exit code 3

[Frank-Ulrich Sommer via Cygwin]

I seem to have almost excatly the same problem except that I could not solve it 
by removing the Cygwin-X folder. In this case during the reinstallation of the 
xinit package the folder is recreated again and then the original error message 
(xinit.sh exit code 3) reappears. The directory again has strange permissions  
when checked with Windows Explorer and I am not allowed the enter it or see its 
contents before resetting the security settings.

When doing an "ls -l" (within Cygwin) in the "Start Menu" folder the group and 
owner for the Cygwin-X directory seem to be reversed compared to other folders manually created 
from Windows Explorer (i.e. the user name appears in the group column and vice versa) but I'm not 
sure if this is important:

d---rwxr-x+ 1 myusername Administratoren    0 May 10 02:27  Cygwin-X

For all other folders the group is displayed before the username.

I had to fix the security settings for the Cygwin-X folder and then manually execute the last two 
"mkshortcut" commands from "/etc/postinstall/xinit.sh" (replacing $CYGWINFORALL with 
"-A" and ${wow64} with an empty string).

Should this be the only problem and should my "fix" be correct? And is there 
anything I can do to help find the cause for this problem?



On 23.10.2023 17:41, Brian Inglis via Cygwin wrote:

On 2023-10-23 06:05, Fergus Daly via Cygwin wrote:

<< Detail >>


When I used Explorer to visit C:\ProgramData\Microsoft\Windows\Start 
Menu\Cygwin-X I was told:
"You don't currently have permission to access this folder"
and clicking on Continue to get access I was told:
"You have been denied permission to access this folder"
There was then offered an option to edit Permissions which I didn't feel like 
pursuing.
(I am the Administrator on my own standalone Windows machine. The denial of 
access to Cygwin-X feels odd.
PS I also have Cygwin32 installed and running. I _am_ permitted access to the 
equivalent folder Cygwin-X (32-bit).)



Please try running the following command/s, under Cygwin 32 and 64, and posting
the outputs:



$ for p in "`cygpath -A -P -U`"{,/Cygwin-X}; do for c in 'lsattr -d' 'ls -dl'
getfacl; do $c "$p"; echo; done; icacls "`cygpath -m "$p"`"; done


Thank you. (Again.)
1. Actually before reading this I had deleted both folders
(successfully, despite not being permitted entry into one
of them) and the re-ran the xinit installation with no
bother at all.
I'm guessing the Permissions glitch resulted from some local
recent accidental keypress or sequence.
2. icacls? Haven't got this though I have got getfacl; found icacl in
"Search packages" under libattica-devel and ng-spice-debuginfo?


$ /proc/cygdrive/c/WINDOWS/system32/icacls /?




--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [users@httpd] Multi site SSL problems

[Frank Gingras]

On Thu, May 9, 2024 at 6:54 PM Chris me  wrote:

> Hi, I am having an issue trying to get multiple sites with their own SSL
> cert. I purchased AlphaSSL certs for them.
>
> The strange thing, the first cert works, the second gives me an
> ERR_SSL_PROTOCOL_ERROR, but only on some systems.
>
>
>
> This is what I am using now:
>
>
>
> (
>
> Site1 is fine, Site2 gives me the error.
>
>
>
> I originally tried with NameVirtualHost *.443
>
> And then 
>
> But when I go to site2, it complains that the cert is invalid because it
> is using the cert from site1?
>
> )
>
>
>
>
>
> 
>
> NameVirtualHost 192.99.9.188:443
>
>
>
> 
>
> ServerName www.site1.com
>
> ServerAdmin webmas...@site1.com
>
> DocumentRoot /home/httpd/sites/site1
>
> 
>
>
>
> Order allow,deny
>
> Allow from all
>
> 
>
>
>
> SSLEngine on
>
> SSLProtocol all -SSLv2 -SSLv3
>
> SSLCertificateFile/etc/ssl/site1.ca/server.crt
>
> SSLCertificateKeyFile /etc/ssl/site1.ca/server.key
>
> SSLCertificateChainFile /etc/ssl/site1.ca/bundle.crt
>
> 
>
>
>
> 
>
> ServerName www.site2.com
>
> ServerAdmin webmas...@site2.com
>
> DocumentRoot /home/httpd/sites/site2
>
> 
>
>
>
> Order allow,deny
>
> Allow from all
>
> 
>
>
>
> SSLEngine on
>
> SSLProtocol all -SSLv2 -SSLv3
>
> SSLCertificateFile/etc/ssl/site2.ca/server.crt
>
> SSLCertificateKeyFile /etc/ssl/site2.ca/server.key
>
> SSLCertificateChainFile /etc/ssl/site2.ca/bundle.crt
>
> 
>
> 
>

So many red flags here:

- Always use *:PORT when defining a vhost, unless you know exactly what you
are doing
- Set the ServerName directive in every single vhost
- Do not use the 2.2 authz directives (Allow/Deny/Order) and use Require
instead
- Unload the mod_access_compat module when apachectl configtest passes

Lastly, show the output from apachectl -S when the fixes are applied

Re: [PATCH] Fix 'make coverage' when used with lcov version 2.0+

[Frank Ch. Eigler]

Hi -

> > > What's the purpose of sending proposed patches to the mailing list
> > > if reviews are silently ignored?
> > 
> > Please be collegial and don't exaggerate.
> 
> The fact is that the review was silently ignored, which is, from my point
> of view, an extraordinary event, 

The review *singular*.  And you can't know whether it was ignored, or
never received or missed or considered.

> [...] and request that the issue pointed out in the review to be
> properly addressed.

May I suggest asking for that in a less accusatory and exaggerated
fashion next time.  Note also that being "properly addressed" is up to
the discretion of the maintainers - one of whom is Aaron.

- FChE

Re: [PATCH] Fix 'make coverage' when used with lcov version 2.0+

[Frank Ch. Eigler]

Hi -

On Fri, May 10, 2024 at 12:53:39AM +0300, Dmitry V. Levin wrote:
> > Pushed as commit ca8ad4648197
> 
> What's the purpose of sending proposed patches to the mailing list
> if reviews are silently ignored?

Please be collegial and don't exaggerate.

- FChE

[Bug 2063456] Re: package cephadm: dependency "cephadmlib" missing

[Frank Barton]

Any word on a formal fix for this hitting the repo?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2063456

Title:
  package cephadm: dependency "cephadmlib" missing

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2063456/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[docker-dev] Buy best description with credit clone card/how to use credit clone card

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•  HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•  HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

[docker-dev] BUY WAY HIGH BALANCE CREDIT CLONE CARD WITH OPEN PIN BALANCE K

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

• HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain 

[docker-dev] Buy no pin credit clone card with low balance K/ fast delivery agency on time

[Frank Domm]

Credit card cloning refers to making an unauthorized copy of a credit card. 
This practice is also sometimes called skimming. Thieves copy information 
at a credit card terminal using an electronic device and transfer the data 
from the stolen card to a new card or rewrite an existing card with the 
information.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•  HOW CREDIT CARD CLONING WORKS 
>From the perspective of the thieves, cloning can be a very effective way to 
obtain credit card information, because it does not require the physical 
credit card to be stolen. Instead, they simply use an electronic device to 
covertly scan the card's information and copy it into the device’s memory. 
The thieves can then access that information digitally, or else download 
the information onto a separate credit card that is already in their 
possession.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•CAN A CLONED CARD BE USED AT AN ATM?
   This cloned data can then be transferred to another card, creating a 
duplicate. If the criminal also has the card's PIN (personal identification 
number), they can use the cloned card to withdraw money from the 
cardholder's account at an ATM.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

•  HOW TO USE THE CLONED CARD'S AT THE ATM MACHINE :
Step 1: Insert ATM Card.
Step 2: Select the Language.
Step 3: Enter 4 Digit ATM Pin.
Step 4: Select Your Transaction.
Step 5: Select Your Account.
Step 6: Enter the Withdrawal Money(withdraw £500 every after 2 hours)
Step 7: Collect the Cash.
Step 8: Take a Printed Receipt.
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary

Once the information is recorded it can be transferred onto the magnetic 
strip of a new card or can be used to overwrite data on an already stolen 
credit card. For cards that use a personal identification number (PIN) 
number in addition to a magnetic strip, such as debit cards, the PIN would 
need to be observed and recorded. This is sometimes difficult to 
accomplish, adding additional protection against having your card 
compromised.

https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
https://t.me/psychdelicmushroomhempdispensary
Search Frequently Asked Questions. Skimming or cloning cards is when the 
details of your card from the magnetic strip are put onto a blank card and 
then the card is used without your knowledge or permission.

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
 Sell Fullz info ssn dob dl Fresh And Good - Sell DL Scan Front, Back + SSN 
number USA-Cvv usa-uk-au-eu-inter


HELLO CLIENT MINE

Tele Channel : https://t.me/psychdelicmushroomhempdispensary

Fullz Info Store

PHOTO/SCANS DL
Canada SCANS DL
United Kingdom SCANS DL
United States SCANS DL
Germany SCANS DL
Italy SCANS DL

- Fullz info USA

Fullz info USA+SSN
Fullz info USA+DL
Fullz info USA+AN:RN
Fullz info USA+Credit Score
Fullz info ID Scan Front+back+ssn+selfie
Fullz info Business

- Fullz info CA

Fullz info CA+SIN

- Fullz info UK

Tele Channel : https://t.me/psychdelicmushroomhempdispensary
ALL KIND OF FULLZ
UK FULLZ WITH NIN ..
UK FULLZ RANDOM ...
UK FULLZ WITH BANK ( BARCLAYS , HSBC , NATIONWIDE )
UK FULLZ WITH MOBILE NETWORK PROVIDERS (O2 , EE , VODAFONE..)
UK FULLZ DEAD + PHONE PROVIDER
UK FULLZ WITH AGE RANGE ..
UK FULLZ WITH SPECIFIC DOOR..
UK FULLZ WITH VBV
ALL OF THEM ARE FRESH FULLZ DEAD AND NOT RESOLD.
Tele Channel : https://t.me/psychdelicmushroomhempdispensary

---There are also many countries listed below---

Fullz info Argentina https://t.me/psychdelicmushroomhempdispensary
Fullz info Marokko https://t.me/psychdelicmushroomhempdispensary
Fullz info Russia https://t.me/psychdelicmushroomhempdispensary
Fullz info Romania https://t.me/psychdelicmushroomhempdispensary
Fullz info Czechia https://t.me/psychdelicmushroomhempdispensary
Fullz info Greece https://t.me/psychdelicmushroomhempdispensary
Fullz info Hungary https://t.me/psychdelicmushroomhempdispensary
Fullz info Latvia https://t.me/psychdelicmushroomhempdispensary
Fullz info Lithuania https://t.me/psychdelicmushroomhempdispensary
Fullz info Switzerland https://t.me/psychdelicmushroomhempdispensary
Fullz info Cyprus https://t.me/psychdelicmushroomhempdispensary
Fullz info Estonia https://t.me/psychdelicmushroomhempdispensary
Fullz info Netherlands https://t.me/psychdelicmushroomhempdispensary
Fullz info Belgium https://t.me/psychdelicmushroomhempdispensary
Fullz info Norway https://t.me/psychdelicmushroomhempdispensary
Fullz info Sweden https://t.me/psychdelicmushroomhempdispensary
Fullz info Ireland https://t.me/psychdelicmushroomhempdispensary
Fullz info Danmark https://t.me/psychdelicmushroomhempdispensary
Fullz info Spain