Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-30 Thread John Baldwin
On Wednesday, April 30, 2014 10:48:27 am Nathan Whitehorn wrote:
> On 04/30/14 06:36, Ed Maste wrote:
> > On 30 April 2014 01:41, Nathan Whitehorn  wrote:
> >> So, to make universally bootable install media, we have to switch the
> >> default console driver to newcons. This means we need to finish polishing
> >> newcons and need to think about what, exactly, the missing pieces are to
> >> enable it as the default.
> > There's a Newcons wiki page at https://wiki.freebsd.org/Newcons with
> > some details, but I believe the main points are:
> >
> > 1. Documentation.  We currently have no vt(4) man page so one needs to
> > be written, and other pages likely need to be updated.  Other
> > documentation (like the handbook) need to be investigated.
> > 2. Control tools (e.g. kbdmap(1), vidcontrol(1), vidfont(1)) need to
> > be updated.  Alexandr posted a patch for vidcontrol with the subject
> > "RFT vidcontrol for vt(4)"
> > 3. Provide console fonts for vt, equivalent to /usr/share/syscons/fonts.
> > 4. Address some performance issues, especially on certain non-x86 and
> > low-end hardware, and in virtual machines.
> > 5. Address boot console priority.
> >
> 
> Which of these are blockers before turning it on in GENERIC for wider 
> testing? Point #1 is a must, I think. #2 seems potentially destructive: 
> do we want to have to make them work with both syscons and vt? #5 we've 
> worked around for the time being. #4 would be nice, but maybe wider 
> testing helps with it. It's painfully slow on regular x86 hardware as 
> well -- my brand-new Lenovo laptop for instance. But I think we could 
> switch GENERIC on -CURRENT over only after writing some documentation.
> -Nathan

I would prefer having a boot time knob to switch between vt or sc and
being able to include both in the kernel.  The boot time knob could default
to vt when booting from EFI.

-- 
John Baldwin
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-30 Thread Nathan Whitehorn

On 04/30/14 06:36, Ed Maste wrote:

On 30 April 2014 01:41, Nathan Whitehorn  wrote:

So, to make universally bootable install media, we have to switch the
default console driver to newcons. This means we need to finish polishing
newcons and need to think about what, exactly, the missing pieces are to
enable it as the default.

There's a Newcons wiki page at https://wiki.freebsd.org/Newcons with
some details, but I believe the main points are:

1. Documentation.  We currently have no vt(4) man page so one needs to
be written, and other pages likely need to be updated.  Other
documentation (like the handbook) need to be investigated.
2. Control tools (e.g. kbdmap(1), vidcontrol(1), vidfont(1)) need to
be updated.  Alexandr posted a patch for vidcontrol with the subject
"RFT vidcontrol for vt(4)"
3. Provide console fonts for vt, equivalent to /usr/share/syscons/fonts.
4. Address some performance issues, especially on certain non-x86 and
low-end hardware, and in virtual machines.
5. Address boot console priority.



Which of these are blockers before turning it on in GENERIC for wider 
testing? Point #1 is a must, I think. #2 seems potentially destructive: 
do we want to have to make them work with both syscons and vt? #5 we've 
worked around for the time being. #4 would be nice, but maybe wider 
testing helps with it. It's painfully slow on regular x86 hardware as 
well -- my brand-new Lenovo laptop for instance. But I think we could 
switch GENERIC on -CURRENT over only after writing some documentation.

-Nathan
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-30 Thread Ed Maste
On 30 April 2014 01:41, Nathan Whitehorn  wrote:
>
> So, to make universally bootable install media, we have to switch the
> default console driver to newcons. This means we need to finish polishing
> newcons and need to think about what, exactly, the missing pieces are to
> enable it as the default.

There's a Newcons wiki page at https://wiki.freebsd.org/Newcons with
some details, but I believe the main points are:

1. Documentation.  We currently have no vt(4) man page so one needs to
be written, and other pages likely need to be updated.  Other
documentation (like the handbook) need to be investigated.
2. Control tools (e.g. kbdmap(1), vidcontrol(1), vidfont(1)) need to
be updated.  Alexandr posted a patch for vidcontrol with the subject
"RFT vidcontrol for vt(4)"
3. Provide console fonts for vt, equivalent to /usr/share/syscons/fonts.
4. Address some performance issues, especially on certain non-x86 and
low-end hardware, and in virtual machines.
5. Address boot console priority.
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-29 Thread Nathan Whitehorn

On 04/25/14 06:07, Glen Barber wrote:

On Fri, Apr 25, 2014 at 02:00:13PM +0400, Dmitry Morozovsky wrote:

seconded, till now the most bootable case for machines under my hands were
happy with no-partition at all (fake MBR with 50k@s4, dd from UFS)


I'm working on restoring the MBR, but need some time to test.  If
I can't get a working memstick image with the addition of an EFI
partition for the UEFI boot loader in a few hours, I'll revert this
commit.

Glen



The amd64 release scripts now include variants that build EFI/BIOS dual 
mode images. These should be bootable everywhere: stupid BIOSes that see 
GPT and try to boot with EFI will succeed since there actually is EFI. 
If you try to boot with BIOS, it will also boot that way. (Thanks to 
Marcel for mkimg, by the way, which have made the scripts much cleaner)


These cannot, however, become the defaults for the time being because 
GENERIC uses syscons and syscons does not support EFI. Adding EFI 
loaders to the image means there is an increased likelihood they will be 
used and failing to give a console afterward is not friendly behavior. 
The VT kernel configuration works with both.


So, to make universally bootable install media, we have to switch the 
default console driver to newcons. This means we need to finish 
polishing newcons and need to think about what, exactly, the missing 
pieces are to enable it as the default.

-Nathan
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-25 Thread Glen Barber
On Fri, Apr 25, 2014 at 02:00:13PM +0400, Dmitry Morozovsky wrote:
> seconded, till now the most bootable case for machines under my hands were 
> happy with no-partition at all (fake MBR with 50k@s4, dd from UFS)
> 

I'm working on restoring the MBR, but need some time to test.  If
I can't get a working memstick image with the addition of an EFI
partition for the UEFI boot loader in a few hours, I'll revert this
commit.

Glen



pgpo68kePnyvB.pgp
Description: PGP signature


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-25 Thread Dmitry Morozovsky
On Thu, 24 Apr 2014, Don Lewis wrote:

[snip]

> >> GPT may not be the best choice here. On a number of, in particular, Lenovo
> >> hardware, the BIOS will unconditionally boot with EFI from GPT media. I'm
> >> not sure we want to just swap the set of machines on which this will not
> >> boot. It probably needs to be nested MBR (or straight MBR -- I forget if
> >> that works) until the boot media work with EFI (which should be soon on
> >> -CURRENT).
> > 
> > Noted.  The thing here is that I want to get an EFI GPT partition on the
> > image eventually, which unless I'm missing something obvious, we cannot
> > mix GPT and MBR.
> > 
> > I don't particularly like swapping which machines boot with this hack.
> > Maybe it's time to do a MBR stick for "legacy" boot, and the GPT stick
> > for UEFI and/or "fails-to-boot-DD" case?
> 
> I've got a fairly recent Gigabyte motherboard that refused to boot an
> Ultimate Boot CD memstick which uses MBR.  I found out that I was was
> able to boot and install from a 'dangerously dedicated' FreeBSD 9.x
> memstick. Given that clue, I built a 'dangerously dedicated' Ultimate
> Boot CD memstick, which worked just fine.

seconded, till now the most bootable case for machines under my hands were 
happy with no-partition at all (fake MBR with 50k@s4, dd from UFS)

this situation could drift in the near future though...

-- 
Sincerely,
D.Marck [DM5020, MCK-RIPE, DM3-RIPN]
[ FreeBSD committer: ma...@freebsd.org ]

*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- ma...@rinet.ru ***

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-24 Thread Don Lewis
On 24 Apr, Glen Barber wrote:
> On Thu, Apr 24, 2014 at 07:12:10PM -0700, Nathan Whitehorn wrote:
>> On 04/24/14 18:38, Glen Barber wrote:
>> >Author: gjb
>> >Date: Fri Apr 25 01:38:57 2014
>> >New Revision: 264907
>> >URL: http://svnweb.freebsd.org/changeset/base/264907
>> >
>> >Log:
>> >   Refactor make-memstick.sh to avoid creating the 'dangerously
>> >   dedicated' partition scheme, reported to cause the memstick.img
>> >   to fail to boot.
>> >   Similar to how make-memstick.sh worked on stable/8, use makefs(8)
>> >   to create the actual filesystem.  Then calculate the size of the
>> >   resulting image file, create the GPT partition scheme, then dd(1)
>> >   the filesystem created with makefs(8) to the freebsd-ufs GPT
>> >   partition.
>> >   This was tested on a known-working machine[1] for regression, and
>> >   a known-not-working machine[2] to ensure the boot issue has been
>> >   resolved.
>> >
>> 
>> GPT may not be the best choice here. On a number of, in particular, Lenovo
>> hardware, the BIOS will unconditionally boot with EFI from GPT media. I'm
>> not sure we want to just swap the set of machines on which this will not
>> boot. It probably needs to be nested MBR (or straight MBR -- I forget if
>> that works) until the boot media work with EFI (which should be soon on
>> -CURRENT).
> 
> Noted.  The thing here is that I want to get an EFI GPT partition on the
> image eventually, which unless I'm missing something obvious, we cannot
> mix GPT and MBR.
> 
> I don't particularly like swapping which machines boot with this hack.
> Maybe it's time to do a MBR stick for "legacy" boot, and the GPT stick
> for UEFI and/or "fails-to-boot-DD" case?

I've got a fairly recent Gigabyte motherboard that refused to boot an
Ultimate Boot CD memstick which uses MBR.  I found out that I was was
able to boot and install from a 'dangerously dedicated' FreeBSD 9.x
memstick. Given that clue, I built a 'dangerously dedicated' Ultimate
Boot CD memstick, which worked just fine.

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-24 Thread Glen Barber
On Thu, Apr 24, 2014 at 07:12:10PM -0700, Nathan Whitehorn wrote:
> On 04/24/14 18:38, Glen Barber wrote:
> >Author: gjb
> >Date: Fri Apr 25 01:38:57 2014
> >New Revision: 264907
> >URL: http://svnweb.freebsd.org/changeset/base/264907
> >
> >Log:
> >   Refactor make-memstick.sh to avoid creating the 'dangerously
> >   dedicated' partition scheme, reported to cause the memstick.img
> >   to fail to boot.
> >   Similar to how make-memstick.sh worked on stable/8, use makefs(8)
> >   to create the actual filesystem.  Then calculate the size of the
> >   resulting image file, create the GPT partition scheme, then dd(1)
> >   the filesystem created with makefs(8) to the freebsd-ufs GPT
> >   partition.
> >   This was tested on a known-working machine[1] for regression, and
> >   a known-not-working machine[2] to ensure the boot issue has been
> >   resolved.
> >
> 
> GPT may not be the best choice here. On a number of, in particular, Lenovo
> hardware, the BIOS will unconditionally boot with EFI from GPT media. I'm
> not sure we want to just swap the set of machines on which this will not
> boot. It probably needs to be nested MBR (or straight MBR -- I forget if
> that works) until the boot media work with EFI (which should be soon on
> -CURRENT).

Noted.  The thing here is that I want to get an EFI GPT partition on the
image eventually, which unless I'm missing something obvious, we cannot
mix GPT and MBR.

I don't particularly like swapping which machines boot with this hack.
Maybe it's time to do a MBR stick for "legacy" boot, and the GPT stick
for UEFI and/or "fails-to-boot-DD" case?

Glen



pgpA4NW3T_rtQ.pgp
Description: PGP signature


Re: svn commit: r264907 - in head/release: amd64 i386

2014-04-24 Thread Nathan Whitehorn

On 04/24/14 18:38, Glen Barber wrote:

Author: gjb
Date: Fri Apr 25 01:38:57 2014
New Revision: 264907
URL: http://svnweb.freebsd.org/changeset/base/264907

Log:
   Refactor make-memstick.sh to avoid creating the 'dangerously
   dedicated' partition scheme, reported to cause the memstick.img
   to fail to boot.
   
   Similar to how make-memstick.sh worked on stable/8, use makefs(8)

   to create the actual filesystem.  Then calculate the size of the
   resulting image file, create the GPT partition scheme, then dd(1)
   the filesystem created with makefs(8) to the freebsd-ufs GPT
   partition.
   
   This was tested on a known-working machine[1] for regression, and

   a known-not-working machine[2] to ensure the boot issue has been
   resolved.
   



GPT may not be the best choice here. On a number of, in particular, 
Lenovo hardware, the BIOS will unconditionally boot with EFI from GPT 
media. I'm not sure we want to just swap the set of machines on which 
this will not boot. It probably needs to be nested MBR (or straight MBR 
-- I forget if that works) until the boot media work with EFI (which 
should be soon on -CURRENT).

-Nathan
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"


svn commit: r264907 - in head/release: amd64 i386

2014-04-24 Thread Glen Barber
Author: gjb
Date: Fri Apr 25 01:38:57 2014
New Revision: 264907
URL: http://svnweb.freebsd.org/changeset/base/264907

Log:
  Refactor make-memstick.sh to avoid creating the 'dangerously
  dedicated' partition scheme, reported to cause the memstick.img
  to fail to boot.
  
  Similar to how make-memstick.sh worked on stable/8, use makefs(8)
  to create the actual filesystem.  Then calculate the size of the
  resulting image file, create the GPT partition scheme, then dd(1)
  the filesystem created with makefs(8) to the freebsd-ufs GPT
  partition.
  
  This was tested on a known-working machine[1] for regression, and
  a known-not-working machine[2] to ensure the boot issue has been
  resolved.
  
  Testers:  myself [1], db [2]
  MFC After:1 week
  Sponsored by: The FreeBSD Foundation

Modified:
  head/release/amd64/make-memstick.sh
  head/release/i386/make-memstick.sh

Modified: head/release/amd64/make-memstick.sh
==
--- head/release/amd64/make-memstick.sh Fri Apr 25 01:20:10 2014
(r264906)
+++ head/release/amd64/make-memstick.sh Fri Apr 25 01:38:57 2014
(r264907)
@@ -3,7 +3,7 @@
 # This script generates a "memstick image" (image that can be copied to a
 # USB memory stick) from a directory tree.  Note that the script does not
 # clean up after itself very well for error conditions on purpose so the
-# problem can be diagnosed (full filesystem most likely but ...).
+# problem can be diagnosed.
 #
 # Usage: make-memstick.sh  
 #
@@ -13,11 +13,15 @@
 PATH=/bin:/usr/bin:/sbin:/usr/sbin
 export PATH
 
+BLOCKSIZE=10240
+
 if [ $# -ne 2 ]; then
   echo "make-memstick.sh /path/to/directory /path/to/image/file"
   exit 1
 fi
 
+tempfile="${2}.$$"
+
 if [ ! -d ${1} ]; then
   echo "${1} must be a directory"
   exit 1
@@ -28,21 +32,62 @@ if [ -e ${2} ]; then
   exit 1
 fi
 
-echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab
-makefs -B little -o label=FreeBSD_Install ${2} ${1}
+rm -f ${tempfile}
+echo "/dev/gpt/rootfs / ufs ro,noatime 1 1" > ${1}/etc/fstab
+makefs ${tempfile} ${1}
 if [ $? -ne 0 ]; then
   echo "makefs failed"
   exit 1
 fi
 rm ${1}/etc/fstab
 
+#
+# Use $BLOCKSIZE for transfers to improve efficiency.  When calculating
+# how many blocks to transfer "+ 520" is to account for truncation in the
+# division and to provide ample padding.
+#
+
+filesize=`stat -f "%z" ${tempfile}`
+blocks=$(($filesize / ${BLOCKSIZE} + 520 ))
+dd if=/dev/zero of=${2} bs=${BLOCKSIZE} count=${blocks}
+if [ $? -ne 0 ]; then
+  echo "creation of image file failed"
+  exit 1
+fi
+
 unit=`mdconfig -a -t vnode -f ${2}`
 if [ $? -ne 0 ]; then
   echo "mdconfig failed"
   exit 1
 fi
-gpart create -s BSD ${unit}
-gpart bootcode -b ${1}/boot/boot ${unit}
-gpart add -t freebsd-ufs ${unit}
+
+gpart create -s gpt /dev/${unit}
+if [ $? -ne 0 ]; then
+  echo "GPT creation failed"
+  exit 1
+fi
+gpart add -t freebsd-boot -s 512k /dev/${unit}
+if [ $? -ne 0 ]; then
+  echo "Creating boot partition failed"
+  exit 1
+fi
+gpart bootcode -b ${1}/boot/pmbr -p ${1}/boot/gptboot -i 1 /dev/${unit}
+if [ $? -ne 0 ]; then
+  echo "Writing bootcode failed"
+  exit 1
+fi
+gpart add -t freebsd-ufs -l rootfs /dev/${unit}
+if [ $? -ne 0 ]; then
+  echo "Creating UFS partition failed"
+  exit 1
+fi
+
+dd if=${tempfile} of=/dev/${unit}p2 bs=$BLOCKSIZE conv=sync
+if [ $? -ne 0 ]; then
+  echo "copying filesystem into image file failed"
+  exit 1
+fi
+
 mdconfig -d -u ${unit}
 
+rm -f ${tempfile}

Modified: head/release/i386/make-memstick.sh
==
--- head/release/i386/make-memstick.sh  Fri Apr 25 01:20:10 2014
(r264906)
+++ head/release/i386/make-memstick.sh  Fri Apr 25 01:38:57 2014
(r264907)
@@ -3,7 +3,7 @@
 # This script generates a "memstick image" (image that can be copied to a
 # USB memory stick) from a directory tree.  Note that the script does not
 # clean up after itself very well for error conditions on purpose so the
-# problem can be diagnosed (full filesystem most likely but ...).
+# problem can be diagnosed.
 #
 # Usage: make-memstick.sh  
 #
@@ -13,11 +13,15 @@
 PATH=/bin:/usr/bin:/sbin:/usr/sbin
 export PATH
 
+BLOCKSIZE=10240
+
 if [ $# -ne 2 ]; then
   echo "make-memstick.sh /path/to/directory /path/to/image/file"
   exit 1
 fi
 
+tempfile="${2}.$$"
+
 if [ ! -d ${1} ]; then
   echo "${1} must be a directory"
   exit 1
@@ -28,21 +32,62 @@ if [ -e ${2} ]; then
   exit 1
 fi
 
-echo '/dev/ufs/FreeBSD_Install / ufs ro,noatime 1 1' > ${1}/etc/fstab
-makefs -B little -o label=FreeBSD_Install ${2} ${1}
+rm -f ${tempfile}
+echo "/dev/gpt/rootfs / ufs ro,noatime 1 1" > ${1}/etc/fstab
+makefs ${tempfile} ${1}
 if [ $? -ne 0 ]; then
   echo "makefs failed"
   exit 1
 fi
 rm ${1}/etc/fstab
 
+#
+# Use $BLOCKSIZE for transfers to improve efficiency.  When calculating
+# how many blocks to transfer "+ 520" is to account for truncation in the
+# di